Search

Diffractive membrane imaging systems have been an important development trend for high-orbit satellite cameras owing to their advantages of large aperture, light weight, rapid manufacture, and low cost. However, caused by the cross-coupling effects of diffraction imaging, membrane properties, subaperture stitching, on-orbit disturbances, and other physical factors, lager-aperture space diffractive membrane imaging systems have specific and complex degradation characteristics: the modulation transfer function (MTF) and signal-to-noise ratio (SNR) have more prominent degradation and serious space-variant characteristics over fields of view, with obvious background radiation properties that seriously affect the application of imaging products. To address this problem, this study established a global information transmission model by characterizing the PSF and background radiation in a full field of view to represent the imaging law of an on-orbit system. Aiming at the inverse problem of the information transmission model, we also propose a novel image inversion restoration method for the special degradation characteristics. In particular, the effect of diffraction efficiency is introduced into the inversion restoration method to solve the background radiation problem. Moreover, we innovatively designed matrix regularization parameters to further improve the correction ability of spatial variation. When the diffraction efficiency was experimentally higher than 60% and the mean measured spatial variability was less than 0.2, the proposed method exhibited a satisfactory processing performance, and could improve multiobjective comprehensive processing, such as transfer function compensation, spatial variation correction, and background radiation removal. [ABSTRACT FROM AUTHOR]

The emerging technology of rotating synthetic aperture (RSA) presents a promising solution for the development of lightweight, large-aperture, and high-resolution optical remote sensing systems in geostationary orbit. However, the rectangular shape of the primary mirror and the distinctive imaging mechanism involving the continuous rotation of the mirror lead to a pronounced decline in image resolution along the shorter side of the rectangle compared to the longer side. The resolution also exhibits periodic time-varying characteristics. To address these limitations and enhance image quality, we begin by analyzing the imaging mechanism of the RSA system. Subsequently, we propose a single-image super-resolution method that utilizes a rotated varied-size window attention mechanism instead of full attention, based on the Vision Transformer architecture. We employ a two-stage training methodology for the network, where we pre-train it on images masked with stripe-shaped masks along the shorter side of the rectangular pupil. Following that, we fine-tune the network using unmasked images. Through the strip-wise mask sampling strategy, this two-stage training approach effectively circumvents the interference of lower confidence (clarity) information and outperforms training the network from scratch using the unmasked degraded images. Our digital simulation and semi-physical imaging experiments demonstrate that the proposed method achieves satisfactory performance. This work establishes a valuable reference for future space applications of the RSA system. [ABSTRACT FROM AUTHOR]

The diffractive membrane optical imaging system is an important development trend of ultra large aperture and lightweight space camera. However, related investigations on physics-based diffractive imaging degradation characteristics and corresponding image restoration methods are less studied. In this paper, the model of image quality degradation for the diffraction imaging system is first deduced mathematically based on diffraction theory and then the degradation characteristics are analyzed. On this basis, a novel regularization model of image restoration that contains multiple prior constraints is established. After that, the solving approach of the equation with the multi-norm coexistence and multi-regularization parameters (prior’s parameters) is presented. Subsequently, the space-variant PSF image restoration method for large aperture diffractive imaging system is proposed combined with block idea of isoplanatic region. Experimentally, the proposed algorithm demonstrates its capacity to achieve multi-objective improvement including MTF enhancing, dispersion correcting, noise and artifact suppressing as well as image’s detail preserving, and produce satisfactory visual quality. This can provide scientific basis for applications and possesses potential application prospects on future space applications of diffractive membrane imaging technology. [ABSTRACT FROM AUTHOR]

With the increasing demand for the wide-area refined detection of aircraft targets, remote sensing cameras have adopted an ultra-large area-array detector as a new imaging mode to obtain broad width remote sensing images (RSIs) with higher resolution. However, this imaging technology introduces new special image degradation characteristics, especially the weak target energy and the low signal-to-noise ratio (SNR) of the image, which seriously affect the target detection capability. To address the aforementioned issues, we propose an aircraft detection method for RSIs with low SNR, termed L-SNR-YOLO. In particular, the backbone is built blending a swin-transformer and convolutional neural network (CNN), which obtains multiscale global and local RSI information to enhance the algorithm's robustness. Moreover, we design an effective feature enhancement (EFE) block integrating the concept of nonlocal means filtering to make the aircraft features significant. In addition, we utilize a novel loss function to optimize the detection accuracy. The experimental results demonstrate that our L-SNR-YOLO achieves better detection performance in RSIs than several existing advanced methods. [ABSTRACT FROM AUTHOR]

The novel rotating synthetic aperture (RSA) is a new optical imaging system that uses the method of rotating the rectangular primary mirror for dynamic imaging. It has the advantage of being lightweight, with no need for splicing and real-time surface shape maintenance on orbit. The novel imaging method leads to complex image quality degradation characteristics. Therefore, it is vital to use the image quality improvement method to restore and improve the image quality to meet the application requirements. For the RSA system, a new system that has not been applied in orbit, it is difficult to construct suitable large datasets. Therefore, it is necessary to study and establish the dynamic imaging characteristic model of the RSA system, and on this basis provide data support for the corresponding image super resolution and restoration method through simulation. In this paper, we first analyze the imaging characteristics and mathematically model the rectangular rotary pupil of the RSA system. On this basis, combined with the analysis of the physical interpretation of the blur kernel, we find that the optimal blur kernel is not the point spread function (PSF) of the imaging system. Therefore, the simulation method of convolving the input image directly with the PSF is flawed. Furthermore, the weights of a convolutional neural network (CNN) are the same for each input. This means that the normal convolutional layer is not only difficult to accurately estimate the time-varying blur kernel, but also difficult to adapt to the change in the length–width ratio of the primary mirror. To that end, we propose a blur kernel estimation conditional convolutional neural network (CCNN) that is equivalent to multiple normal CNNs. We extend the CNN to a conditional model by taking an encoding as an additional input and using conditionally parameterized convolutions instead of normal convolutions. The CCNN can simulate the imaging characteristics of the rectangular pupil with different length–width ratios and different rotation angles in a controllable manner. The results of semi-physical experiments show that the proposed simulation method achieves a satisfactory performance, which can provide data and theoretical support for the image restoration and super-resolution method of the RSA system. [ABSTRACT FROM AUTHOR]

The novel rotating synthetic aperture imaging system has the advantages of short processing cycle, light weight, low cost, and convenient transport, which is a potential development direction for future optical satellite payloads with ultra-large aperture. However, influenced by the coupling effects of the rectangular primary mirror, rotating imaging, and link perturbation, images from this system may have unique spatial and temporal variability with resolution anisotropy, resulting in the dynamics and weak features of targets. The insufficient recognition of system characteristics seriously restricts the application ability of corresponding target interpretation methods. To address this problem, we start with the analysis of system imaging characteristics and target interpretation mechanisms. Considering calculating texture, grayscale, and edge feature parameters in evaluation regions of different sizes, we propose a degradation assessment method based on multi-scale feature characterization. Then, combined with the theoretical calculation and practical target detection experiments, the influence of crucial parameters such as the aspect ratio and rotation angle of the primary mirror on the target interpretation capability is analyzed. The results will provide guidance for the integrated design and optimization of optical imaging systems and intelligent image processing methods. • A degradation assessment method based on multi-scale feature characterization is proposed. • The effect of rotating synthetic aperture system on image target characteristic degradation is analyzed. • The impacts of key system parameters on target detection performance are analyzed. [ABSTRACT FROM AUTHOR]

• The imaging mechanism of the novel rotating synthetic aperture system is deduced. • The image degradation model of the rotating synthetic aperture system is presented. • The effects of factors causing image quality degradation are analyzed. The Rotating synthetic aperture (RSA) is one of the most promising novel imaging systems to meet the requirements of lightweight and high resolution for optical remote sensing satellites, especially in geostationary orbit. The point spread function (PSF) orientation relative to the observed scene, intentionally changes to measure all spatial frequency information available to a filled circular aperture having a diameter approximately equal to the length of the rectangular aperture. As with all precision optical systems, RSA is also subject to dynamic disturbances from the spacecraft/environment. If not accommodated properly in the measurement and synthesis process, the dynamic disturbances can result in significant image quality degradation. To quantify methods to reduce or eliminate the errors, we first analyze the on-orbit imaging mechanism of the RSA system theoretically and then represent the worst-case on-orbit imaging degradation characteristics. [ABSTRACT FROM AUTHOR]

• A prior analysis method of remote sensing image characteristics is presented based on hyper-Laplacian regression. • A novel regularization model is established by taking the characteristic prior parameters and image together as the optimization target. • The solving approach is presented based on the alternate iteration of damping least square and iterative jumping thresholding algorithm. • A fine-processing method based on the adaptive hyper-Laplacian prior is put forward, combined with the idea of block processing. High-quality remote sensing images have wide application prospects in agroforestry investigation, target monitoring, disaster prevention, urban planning and military defense. However, remote sensing imaging links such as atmosphere, platform and optical system seriously affect the ability of image interpretation and analysis. The traditional regularized processing methods have a strong ability to improve the definition, but most of them may sacrifice texture details or introduce artifacts, because their fixed prior parameters cannot fully adapt to various kinds of scenes. To address this problem, we propose a novel fine-processing method based on the adaptive hyper-Laplacian prior for remote sensing imaging systems. The method is developed by automatically updating and optimizing the prior parameters and objective function in the iterative process based on the prior characteristics of different regions of remote sensing images. Experimentally, the proposed method can realize the fine-processing of remote sensing images, including the edge enhancement, texture detail preservation, and artifact suppression. [ABSTRACT FROM AUTHOR]

Rotating Synthetic Aperture (RSA) technology is one of the distinctly advantageous Earth geostationary orbit optical remote sensing technologies. However, the continuous rotation of the RSA system's rectangular primary mirror results in a discernible drop in resolution along the shorter side of the mirror. Additionally, the captured images exhibit periodic and time-varying characteristics. To improve the image quality to meet interpretation needs, we first delineate the imaging process of the rotating primary mirror and analyze the characteristics of image degradation based on the system's imaging mechanism. Then, we propose a dual super-resolution (SR) framework based on Swin Transformer and introduce a self-supervised learning method for jointly training the unified SR network using wavelet fusion. The self-supervised learning method effectively utilizes the spatiotemporal correlation of the information contained in images captured at different rotation directions of the rectangular pupil. Moreover, the attention mechanism in Transformer can adopt a global perspective and utilize content-based interactions between image content and attention weights to model strong long-range dependencies in remote sensing images. This approach significantly enhances image quality along the pupil's shorter side, consequently yielding superior results. Extensive digital and semi-physical imaging experiments, involving six aspect ratios of the primary mirror, demonstrate that our SR method surpasses state-of-the-art methods. The work in this paper can serve as a valuable reference for future space applications of the RSA technology. • Analyzing the imaging mechanisms of the rotating synthetic aperture system. • Self-supervised learning enhances the utilization of image internal information. • Designing a dual framework that can be trained jointly for image super resolution. • Interactions between image content and attention weights improve performance. • Performance is evaluated through digital and semi-physical experiments. [ABSTRACT FROM AUTHOR]

• Regularization model with multiple prior constraints and matrixing confidence parameters was established. • The approach to solving the equation with multi-norm coexistence was presented. • The effect of diffraction efficiency on restored images was analyzed. We proposed a novel inversion restoration method for the newly developing space diffractive membrane imaging system, in which specific imaging characteristics were first integrated into the degradation model. On this basis, a novel image inversion restoration model was established based on sparse regularization frames and matrixing confidence parameters. Then, an efficient solution method was proposed using an improved iterative shrinkage-thresholding algorithm. Experimentally, when the diffraction efficiency was higher than 60%, the proposed inversion restoration method exhibited a satisfactory processing performance, and could achieve multi-objective image quality improvement, including texture detail restoration, background radiation removal, and noise suppression. [ABSTRACT FROM AUTHOR]

• Analyzing the imaging characteristics of the rotating synthetic aperture system. • Intra-frame self-attention enhances the utilization of spatial information. • Inter-frame mutual attention enhances the utilization of temporal information. • Interactions between image content and attention weights improve performance. • Performance is evaluated through digital and semi-physical experiments. Rotating synthetic aperture (RSA) technology offers a promising solution for achieving large-aperture and lightweight designs in optical remote-sensing systems. It employs a rectangular primary mirror, resulting in noncircular spatial symmetry in the point-spread function, which changes over time as the mirror rotates. Consequently, it is crucial to employ an appropriate image-fusion method to merge high-resolution information intermittently captured from different directions in the image sequence owing to the rotation of the mirror. However, existing image-fusion methods have struggled to address the unique imaging mechanism of this system and the characteristics of the geostationary orbit in which the system operates. To address this challenge, we model the imaging process of a noncircular rotating pupil and analyse its on-orbit imaging characteristics. Based on this analysis, we propose an image-fusion network based on a vision transformer. This network incorporates inter-frame mutual attention and intra-frame self-attention mechanisms, facilitating more effective extraction of temporal and spatial information from the image sequence. Specifically, mutual attention was used to model the correlation between pixels that were close to each other in the spatial and temporal dimensions, whereas long-range spatial dependencies were captured using intra-frame self-attention in the rotated variable-size attention block. We subsequently enhanced the fusion of spatiotemporal information using video swin transformer blocks. Extensive digital simulations and semi-physical imaging experiments on remote-sensing images obtained from the WorldView-3 satellite demonstrated that our method outperformed both image-fusion methods designed for the RSA system and state-of-the-art general deep learning-based methods. 1 1 Dataset is available at https://github.com/sherlockyusun/RSA-Fuse. [ABSTRACT FROM AUTHOR]

The rotating synthetic aperture (RSA) optical imaging system employs a rectangular primary mirror for detection. During the imaging process, the primary mirror rotates around the center to achieve the aperture equivalent to the long side of the rectangle at different rotation angles. As a result, the system's point spread function changes over time, causing periodic time-varying characteristics in the acquired images' resolution. Moreover, due to the rectangular primary mirror, the images obtained by the RSA system are spatially asymmetric, with a lower resolution in the short side's direction than in the long side's direction. Hence, image processing techniques are necessary to enhance the image quality. To provide reference for the study of image quality improvement methods, we first characterize the imaging quality degradation mechanism of the RSA system and the time–space evolution law of the imaging process. We then establish an imaging experiment platform to simulate the dynamic imaging process of the RSA system. We quantify the RSA system's impact on image degradation using objective indexes. Subsequently, by comparing the imaging experiment results with theoretical analysis, we verify the spatially asymmetric and temporally periodic imaging characteristics of the RSA system. Lastly, we introduce image super-resolution experiments to assess the limitations of directly applying generic deep learning-based single image super-resolution methods to the images captured by the RSA system, thereby revealing the challenges involved in improving image quality for the RSA system. [ABSTRACT FROM AUTHOR]

The rotating synthetic aperture (RSA) optical imaging system employs a rectangular primary mirror for detection. During the imaging process, the primary mirror rotates around the center to achieve the aperture equivalent to the long side of the rectangle at different rotation angles. As a result, the system's point spread function changes over time, causing periodic time-varying characteristics in the acquired images' resolution. Moreover, due to the rectangular primary mirror, the images obtained by the RSA system are spatially asymmetric, with a lower resolution in the short side's direction than in the long side's direction. Hence, image processing techniques are necessary to enhance the image quality. To provide reference for the study of image quality improvement methods, we first characterize the imaging quality degradation mechanism of the RSA system and the time–space evolution law of the imaging process. We then establish an imaging experiment platform to simulate the dynamic imaging process of the RSA system. We quantify the RSA system's impact on image degradation using objective indexes. Subsequently, by comparing the imaging experiment results with theoretical analysis, we verify the spatially asymmetric and temporally periodic imaging characteristics of the RSA system. Lastly, we introduce image super-resolution experiments to assess the limitations of directly applying generic deep learning-based single image super-resolution methods to the images captured by the RSA system, thereby revealing the challenges involved in improving image quality for the RSA system. [ABSTRACT FROM AUTHOR]

The growing intelligence and prevalence of drones have led to an increase in their disorderly and illicit usage, posing substantial risks to aviation and public safety. This paper focuses on addressing the issue of drone detection through surveillance cameras. Drone targets in images possess distinctive characteristics, including small size, weak energy, low contrast, and limited and varying features, rendering precise detection a challenging task. To overcome these challenges, we propose a novel detection method that extends the input of YOLOv5s to a continuous sequence of images and inter-frame optical flow, emulating the visual mechanisms employed by humans. By incorporating the image sequence as input, our model can leverage both temporal and spatial information, extracting more features of small and weak targets through the integration of spatiotemporal data. This integration augments the accuracy and robustness of drone detection. Furthermore, the inclusion of optical flow enables the model to directly perceive the motion information of drone targets across consecutive frames, enhancing its ability to extract and utilize features from dynamic objects. Comparative experiments demonstrate that our proposed method of extended input significantly enhances the network's capability to detect small moving targets, showcasing competitive performance in terms of accuracy and speed. Specifically, our method achieves a final average precision of 86.87%, representing a noteworthy 11.49% improvement over the baseline, and the speed remains above 30 frames per second. Additionally, our approach is adaptable to other detection models with different backbones, providing valuable insights for domains such as Urban Air Mobility and autonomous driving. [ABSTRACT FROM AUTHOR]

The vehicle detection in remote sensing images is a challenging task due to the small size of the objects and interference of a complex background. Traditional methods require a large number of anchor boxes, and the intersection rate between these anchor boxes and an object's real position boxes needs to be high enough. Moreover, the size and aspect ratio of each anchor box need to be designed manually. For small objects, more anchor boxes need to be set. To solve these problems, we regard the small object as a keypoint in the relevant background and propose an anchor-free vehicle detection network (AVD-kpNet) to robustly detect small-sized vehicles in remote sensing images. The AVD-kpNet framework fuses features across layers with a deep layer aggregation architecture, preserving the fine features of small objects. First, considering the correlation between the object and the surrounding background, a 2D Gaussian distribution strategy is adopted to describe the ground truth, instead of a hard label approach. Moreover, we redesign the corresponding focus loss function. Experimental results demonstrate that our method has a higher accuracy for the small-sized vehicle detection task in remote sensing images compared with several advanced methods. [ABSTRACT FROM AUTHOR]