Search

Contrastive learning is a self-supervised comparison of two samples to identify characteristics and traits that distinguish one data class from another, improving performance on visual tasks. The performance of existing super-resolution-based approaches degrades with increasing scaling factors, hence practically not useful for high-resolution (HR) imaging applications. We proposed a novel framework that uses contrastive training followed by a decoder to generate an “Artificial style image,” which is utilized as a style image for neural style transfer (NST) learning for image super-resolution in an unsupervised manner. The idea is to benefit from HR textures and features as a style and transfer on an original low-resolution (LR) content image as base elements. The proposed framework has three benefits: 1) the framework is capable of super-resolving different modalities of data like single-band remote sensing images, multispectral band images, RGB remote sensing images, and real-world natural images; 2) proposed method outperforms existing unsupervised and also supervised learning-based methods for both visual and qualitative performance; and 3) leveraging NST learning for remote sensing image super-resolution is performed without sacrificing speed and resources. The framework is novel since the work on NST learning to super-resolve remote sensing images in an unsupervised manner has yet to be acknowledged.

Infrared (IR) imagery sequences are commonly used for detecting moving targets in the presence of evolving cloud clutter or background noise. This research focuses on slow-moving point targets that are less than one pixel in size, such as aircraft at long range from a sensor. Since transmitting IR imagery sequences to a base unit or storing them consumes considerable time and resources, a compression method that maintains the point target detection capabilities is highly desirable. In this work, we introduce a new parametric temporal compression that incorporates Gaussian fit and polynomial fit. We then proceed to spatial compression by spatially applying the lowest possible number of bits for representing each parameter over the parameters extracted by temporal compression, which is followed by bit encoding to achieve an end-to-end compression process of the sequence for data storage and transmission. We evaluate the proposed compression method using the variance estimation ratio score (VERS), which is a signal-to-noise ratio (SNR)-based measure for point target detection that scores each pixel and yields an SNR scores image. A high pixel score indicates that a target is suspected to traverse the pixel. From this score image we calculate the movie scores, which are found to be close to those of the original sequences. Furthermore, we present a new algorithm for automatic detection of the target tracks. This algorithm extracts the target location from the SNR scores image, which is acquired during the evaluation process, using Hough transform. This algorithm yields a similar detection probability (PD) and false alarm probability (PFA) of the compressed sequences and the original sequences. The parameters of the new parametric temporal compression successfully differentiate the targets from the background, yielding high PDs (above 83%) with low PFAs (below 0.043%) without the need to calculate pixel scores or to apply automatic detection of the target tracks.

Infrared (IR) imagery sequences are commonly used for detecting moving targets in the presence of evolving cloud clutter or background noise. This research concentrates on slow-moving point targets that are less than one pixel in size, such as aircraft at long ranges from a sensor. Because transmitting IR imagery sequences to a base unit or storing them consumes considerable time and resources, a compression method that maintains the point-target detection capabilities is highly desirable. In our previous work, we introduced two temporal compression methods that preserve the temporal profile properties of the point target in the form of discrete cosine transform (DCT) quantization and parabola fit. In the present work, we extend the compression task method of DCT quantization by applying spatial compression over the temporally compressed coefficients, which is followed by bit encoding. We evaluate the proposed compression method using a signal-to-noise ratio (SNR)–based measure for point target detection and find that it yields better results than the compression standard H.264. Furthermore, we introduce an automatic detection algorithm that extracts the target location from the SNR scores image, which is acquired during the evaluation process and has a probability of detection and a probability of false alarm close to those of the original sequences. We previously determined that it is necessary to establish a minimal noise level in the SNR-based measure to compensate for smoothing that is induced by the compression. Here, the noise level calculation process is modified in order to allow detection of targets traversing all background types.

During the last decades, statistical models, such as the Ising model, have become very useful in describing solid state systems. These models excel in their simplicity and versatility. Furthermore, their results get quite often accurate experimental proofs. Leading researchers have used them successfully during the last years to restore images. A simple method, based on the Ising model, was used recently in order to restore B/W and grayscale images and achieved preliminary results. In this paper we outline first the analogy between statistical physics and image processing. Later, we present the results we have achieved using a similar, though more complex iterative model in order to get a better restoration. Moreover, we describe models which enable us to restore colored images. Additionally, we present the results of a novel method in which similar algorithms enable us to restore degraded video signals. We confront our outcomes with the results achieved by the simple algorithm and by the median filter for various kinds of noise. Our model reaches PSNR values which are 2-3 dB higher, and SSIM values which are 15%-20% higher than the results achieved by the median filter for video restoration.

Infrared imagery sequences are used for detecting moving targets in the presence of evolving cloud clutter or background noise. This research concentrates on slow-moving point targets that are less than one pixel in size, such as aircraft at long ranges from a sensor. The infrared (IR) imagery sequences that are captured by ground sensors contain an enormous amount of data. Since transmitting this data to a base unit or storing it consumes considerable time and resources, a compression method that maintains the point target detection capabilities is desired. For this purpose, we developed two temporal compression methods that preserve the temporal profile properties of the point target. We evaluated the proposed compression methods using a signal-to-noise-ratio (SNR)-based measure for point target detection and showed that the compression may improve the SNR results compared to the IR sequence prior to compression.

Transmission of a compressed video signal over a lossy communication network exposes the information to losses and errors, which leads to significant visible errors in the reconstructed frames at the decoder side. In this paper we present a new hybrid error concealment algorithm for compressed video sequences, based on temporal and spatial concealment methods. We describe spatial and temporal techniques for the recovery of lost blocks. In particular, we develop postprocessing techniques for the reconstruction of missing or damaged macroblocks. A new decision support tree is developed to efficiently choose the best appropriate error concealment method, according to the spatial and temporal characteristics of the sequence. The proposed algorithm is compared to three error concealment methods: spatial, temporal, and a previous hybrid approach using different noise levels. The results are evaluated using four quality measures. We show that our error concealment scheme outperforms all the other three methods for all the tested video sequences.

We evaluate the degradation in a video sequence that suffers from increasing packet loss ratio (PLR), by using several kinds of quality measures, namely, mean square error (MSE); Amax(the root MSE, which is weighted by the maximum signal change between pairs of pixels—the current pixel and eight surrounding neighbors); the video perceptual distortion measure (VPDM); and a perceptual distortion metric for digital color video (VPDM2). Packet losses can occur at different positions in the compressed video sequence, for example, in headers, Iframes, Pframes, and Bframes. Each packet loss results in a different amount of distortion in the received video signal. We demonstrate the differences between two packet loss models—IID (identical independent distribution) and burst (a sequence of packets lost together). We also examine the influence of packet loss in a communication network on MPEG2 compressed digital video quality. Three of these measures emulate features of the human visual system: the VPDM, which considers temporal masking of distortion (the human eye notices less degradation within a frame, if it appears during fast changes between consequent frames, such as scene cuts); the Amax,which encompasses spatial masking, resulting in small objects within high-frequency neighborhoods being less noticeable; and the VPDM2, which takes into account the human vision color channels and temporal and spatial masking. The results point out that loss due to burst degrades the quality of the compressed video stream significantly less than loss due to IID. The video quality can still be tolerable for a 0.1 PLR using the burst loss model, but for the IID loss model, even for lower PLRs (i.e., 0.01), the video is not perceptible to the human viewer. © 2004 Society of Photo-Optical Instrumentation Engineers.

Time-delay and integration (TDI) arrays are of great interest, especially in connection with civilian applications such as document scanning and industrial product inspection. We consider here a staggered TDI system, in which the even field is delayed with respect to the odd field by 29 pixels. Because of cooling system vibrations and the field differences, pictures become distorted as a result of motion blur. The proposed algorithm improves image quality. We build an analytical model of this system to describe all motion in the picture, including object motion in addition to cooling system vibrations, or either type of motion. We present a number of approaches that can be useful to solve such problems. These solutions are based on different applications of a block-matching algorithm (BMA). Full-search BMA yields the lowest complexity but not necessarily the best image quality. Interpolation BMA yields the best image quality. © 2003 Society of Photo-Optical Instrumentation Engineers.

Many properties of the atmosphere affect the quality of images propagating through it by blurring and reducing their contrast. The atmospheric path involves several limitations, such as scattering, absorption of light, and turbulence, which degrade the image. Recently developed atmospheric filters, which correct for turbulence blur, aerosol blur, and path radiance simultaneously, are implemented here in the digital restoration of Landsat thematic mapper (TM) imagery. The turbulence modulation transfer function (MTF) is calculated from meteorological data or estimated if no meteorological data were measured. Aerosol MTF is consistent with optical depth. The product of the two yields atmospheric MTF, which is implemented in the atmospheric filter. Restoration improves smallness of size of both resolvable detail and contrast. Restorations are quite apparent even under clear weather conditions. A way to evaluate restoration improvement is presented here by the use of quantitative criteria as well as subjective opinions of human observers in perception experiments. Not all the restoration criteria represent improvement in the same tested image under the same restoration conditions. When one criterion suggests an enhancement, there is a chance that another one might represent a lower value for restoration success. © 2003 Society of Photo-Optical Instrumentation Engineers.

We investigate the influence of motion sensor errors on the calculation of a modulation transfer function (MTF) and its implementation in image restoration. The goal of this research is to describe an automatic system for restoration of pictures blurred by vibration, and to consider its possible disadvantages. We present an analytical approach for estimating the vibration MTF from the measured system MTF according to the frequency response of motion sensors and their noise data. Our method is based on point-spread function verification by the data of motion sensor characteristics. We build an analytical model of the sensor and analyze influences of errors caused by system noise and incorrect axis direction of the restoration device. Some image restoration of degraded images is presented based on improvements using the original wiener filter. Performances of inverse and wiener filter operations are compared, and the dependence of restoration quality on the signal-to-noise ratio and angle between the restoration axis and true vibration direction is considered. We also demonstrate the axis rotation sensitivity of the system, and describe improvements over the initial method as seen from our simulation. The key to the restoration is the determination of the improved modulation transfer function unique to the image vibration and sensor characteristics. © 2002 Society of Photo-Optical Instrumentation Engineers.

When an ATM node discovers that it cannot continue the setup of a virtual channel under the requested Quality of Service (QoS), it initiates a backtracking procedure called “crankback.” We propose a novel scheme, referred to as crankback prediction, that decreases the crankback overhead. Under the proposed scheme, nodes check during the connection admission control procedure whether the establishment of a virtual channel has a good chance to be admitted over the entire designated route. If this is not the case, crankback is initiated even before a particular QoS parameter is violated. The main idea behind the proposed scheme is to allocate a “quota” to the Peer Groups (PGs) along the message path, and then to suballocate this quota to the child PGs of these PGs. This process continues recursively until reaching the 1-level PG, which contains only physical nodes. The main advantage of the proposed scheme is that it lowers the setup delay and the processing and communication load imposed by signaling messages that establish unused portions of Virtual Channels (VCs)

One result of the recent advances in different components of imaging systems technology is that these systems have become more resolution-limited and less noise-limited. The most useful tool utilized in characterization of resolution-limited systems is the modulation transfer function (MTF). Our goal is to use the MTF as an image quality measure of image compression implemented by the JPEG (Joint Photographic Experts Group) algorithm and transmitted MPEG (Motion Picture Experts Group) compressed video stream through a lossy packet network. Although we realize that the MTF is not an ideal parameter with which to measure image quality after compression and transmission due to the nonlinearity shift-variant process, we examine the conditions under which it can be used as an approximated criterion for image quality. The advantage of using the MTF of the compression algorithm is that it can be easily combined with the overall MTF of the imaging system. © 2001 Society of Photo-Optical Instrumentation Engineers.

An enhancement of the Piecewise Constant Rate Transmission and Transport (PCRTT) algorithm for reducing the burstiness of a video stream, based on smoothing constant interval is proposed. The new algorithm, called e-PCRTT, relies on geometrical consideration rather than traditional rate-control analysis. E-PCRTT is shown to construct transmission rate-plans with smaller buffer sizes, as compared to the original PCRTT. Alternatively, for the same buffer size, e-PCRTT reduces the number of bandwidth changes as compared to PCRTT. In addition, e-PCRTT produces a rate-plan that requires a smaller initial playback delay.

We present an algorithm that improves the overall performance of compression algorithms by applying low-pass filtering before the compression process and inverse filtering after decompression. Prefiltering images prior to encoding removes high spatial frequencies from the original image, thus increasing the compression ratio of the encoder. The image degradation caused by the filter together with the nonlinear transformation of a typical compression algorithm reduces the entropy of the original image, thus increasing the compression ratio. In addition to improving the compression ratio, the suggested scheme reduces the blocking effect that is common to block-based compression algorithms, such as the Joint Picture Experts Group (JPEG). © 2001 Society of Photo-Optical Instrumentation Engineers.

The image quality resulting from a 2-D image-sampling process by an array of pixels is described. The description is based on a Fourier transformation of the Wigner-Seitz cell, which transforms a unit cell of the sampling lattice in the spatial domain into a bandwidth cell in the spatial-frequency domain. The area of the resulting bandwidth cell is a quantitative measure of the image fidelity of the sampling process. We compare the image-quality benefits of three different oversampling geometries in terms of the modulation transfer function (MTF) as a function of the amount of oversampling used. © 1999 Society of Photo-Optical Instrumentation Engineers.

Any image acquired by optical, electro-optical, or electronic means is likely to be degraded by the environment. The resolution of the acquired image depends on the total modulation transfer function (MTF) of the system and the additive noise. Image restoration techniques can improve image resolution significantly; however, as the noise increases, improvements via image processing become more limited because image restoration increases the noise level in the image. We characterize the influence of the MTF and noise level on human target acquisition probability to ascertain the advantages, if any, of image restoration. Conditions when restoration would be advisable are determined. © 1998 Society of Photo-Optical Instrumentation Engineers.

We analyze the connection between viewer-perceived quality and encoding schemes. The encoding schemes depend on transmission bit-rate, MPEG compression depth, frame size and frame rate in a constant bit-rate (CBR) video transmission of a MPEG-2 video sequence. The compressed video sequence is transmitted over a lossy communication network with quality of service (QoS) and a certain Internet (IP) loss model. On the end-user side, viewer-perceived quality depends on changes in the network conditions, the video compression, and the video content complexity. We demonstrate that, when jointly considering the impact of coding bit rate, packet loss, and video complexity, there is an optimal encoding scheme, which also depends on the video content. We use a set of subjective tests to demonstrate that this optimal encoding scheme maximizes the viewer-perceived quality.