Search

Compared with anchor-based detectors, anchor-free detectors have the advantage of flexibility and a lower calculation complexity. However, in complex remote sensing scenes, the limited geometric size, weak features of objects, and widely distributed environmental elements similar to the characteristics of objects make small object detection a challenging task. To solve these issues, we propose an anchor-free detector named FE-CenterNet, which can accurately detect small objects such as vehicles in complicated remote sensing scenes. First, we designed a feature enhancement module (FEM) composed of a feature aggregation structure (FAS) and an attention generation structure (AGS). This module contributes to suppressing the interference of false alarms in the scene by mining multiscale contextual information and combining a coordinate attention mechanism, thus improving the perception of small objects. Meanwhile, to meet the high positioning accuracy requirements of small objects, we proposed a new loss function without extra calculation and time cost during the inference process. Finally, to verify the algorithm performance and provide a foundation for subsequent research, we established a dim and small vehicle dataset (DSVD) containing various objects and complex scenes. The experiment results demonstrate that the proposed method performs better than mainstream object detectors. Specifically, the average precision (AP) metric of our method is 7.2% higher than that of the original CenterNet with only a decrease of 1.3 FPS. [ABSTRACT FROM AUTHOR]

The novel rotating synthetic aperture imaging system has the advantages of short processing cycle, light weight, low cost, and convenient transport, which is a potential development direction for future optical satellite payloads with ultra-large aperture. However, influenced by the coupling effects of the rectangular primary mirror, rotating imaging, and link perturbation, images from this system may have unique spatial and temporal variability with resolution anisotropy, resulting in the dynamics and weak features of targets. The insufficient recognition of system characteristics seriously restricts the application ability of corresponding target interpretation methods. To address this problem, we start with the analysis of system imaging characteristics and target interpretation mechanisms. Considering calculating texture, grayscale, and edge feature parameters in evaluation regions of different sizes, we propose a degradation assessment method based on multi-scale feature characterization. Then, combined with the theoretical calculation and practical target detection experiments, the influence of crucial parameters such as the aspect ratio and rotation angle of the primary mirror on the target interpretation capability is analyzed. The results will provide guidance for the integrated design and optimization of optical imaging systems and intelligent image processing methods. • A degradation assessment method based on multi-scale feature characterization is proposed. • The effect of rotating synthetic aperture system on image target characteristic degradation is analyzed. • The impacts of key system parameters on target detection performance are analyzed. [ABSTRACT FROM AUTHOR]

The emerging technology of rotating synthetic aperture (RSA) presents a promising solution for the development of lightweight, large-aperture, and high-resolution optical remote sensing systems in geostationary orbit. However, the rectangular shape of the primary mirror and the distinctive imaging mechanism involving the continuous rotation of the mirror lead to a pronounced decline in image resolution along the shorter side of the rectangle compared to the longer side. The resolution also exhibits periodic time-varying characteristics. To address these limitations and enhance image quality, we begin by analyzing the imaging mechanism of the RSA system. Subsequently, we propose a single-image super-resolution method that utilizes a rotated varied-size window attention mechanism instead of full attention, based on the Vision Transformer architecture. We employ a two-stage training methodology for the network, where we pre-train it on images masked with stripe-shaped masks along the shorter side of the rectangular pupil. Following that, we fine-tune the network using unmasked images. Through the strip-wise mask sampling strategy, this two-stage training approach effectively circumvents the interference of lower confidence (clarity) information and outperforms training the network from scratch using the unmasked degraded images. Our digital simulation and semi-physical imaging experiments demonstrate that the proposed method achieves satisfactory performance. This work establishes a valuable reference for future space applications of the RSA system. [ABSTRACT FROM AUTHOR]

The rotating synthetic aperture (RSA) optical imaging system employs a rectangular primary mirror for detection. During the imaging process, the primary mirror rotates around the center to achieve the aperture equivalent to the long side of the rectangle at different rotation angles. As a result, the system's point spread function changes over time, causing periodic time-varying characteristics in the acquired images' resolution. Moreover, due to the rectangular primary mirror, the images obtained by the RSA system are spatially asymmetric, with a lower resolution in the short side's direction than in the long side's direction. Hence, image processing techniques are necessary to enhance the image quality. To provide reference for the study of image quality improvement methods, we first characterize the imaging quality degradation mechanism of the RSA system and the time–space evolution law of the imaging process. We then establish an imaging experiment platform to simulate the dynamic imaging process of the RSA system. We quantify the RSA system's impact on image degradation using objective indexes. Subsequently, by comparing the imaging experiment results with theoretical analysis, we verify the spatially asymmetric and temporally periodic imaging characteristics of the RSA system. Lastly, we introduce image super-resolution experiments to assess the limitations of directly applying generic deep learning-based single image super-resolution methods to the images captured by the RSA system, thereby revealing the challenges involved in improving image quality for the RSA system. [ABSTRACT FROM AUTHOR]

The rotating synthetic aperture (RSA) optical imaging system employs a rectangular primary mirror for detection. During the imaging process, the primary mirror rotates around the center to achieve the aperture equivalent to the long side of the rectangle at different rotation angles. As a result, the system's point spread function changes over time, causing periodic time-varying characteristics in the acquired images' resolution. Moreover, due to the rectangular primary mirror, the images obtained by the RSA system are spatially asymmetric, with a lower resolution in the short side's direction than in the long side's direction. Hence, image processing techniques are necessary to enhance the image quality. To provide reference for the study of image quality improvement methods, we first characterize the imaging quality degradation mechanism of the RSA system and the time–space evolution law of the imaging process. We then establish an imaging experiment platform to simulate the dynamic imaging process of the RSA system. We quantify the RSA system's impact on image degradation using objective indexes. Subsequently, by comparing the imaging experiment results with theoretical analysis, we verify the spatially asymmetric and temporally periodic imaging characteristics of the RSA system. Lastly, we introduce image super-resolution experiments to assess the limitations of directly applying generic deep learning-based single image super-resolution methods to the images captured by the RSA system, thereby revealing the challenges involved in improving image quality for the RSA system. [ABSTRACT FROM AUTHOR]

The growing intelligence and prevalence of drones have led to an increase in their disorderly and illicit usage, posing substantial risks to aviation and public safety. This paper focuses on addressing the issue of drone detection through surveillance cameras. Drone targets in images possess distinctive characteristics, including small size, weak energy, low contrast, and limited and varying features, rendering precise detection a challenging task. To overcome these challenges, we propose a novel detection method that extends the input of YOLOv5s to a continuous sequence of images and inter-frame optical flow, emulating the visual mechanisms employed by humans. By incorporating the image sequence as input, our model can leverage both temporal and spatial information, extracting more features of small and weak targets through the integration of spatiotemporal data. This integration augments the accuracy and robustness of drone detection. Furthermore, the inclusion of optical flow enables the model to directly perceive the motion information of drone targets across consecutive frames, enhancing its ability to extract and utilize features from dynamic objects. Comparative experiments demonstrate that our proposed method of extended input significantly enhances the network's capability to detect small moving targets, showcasing competitive performance in terms of accuracy and speed. Specifically, our method achieves a final average precision of 86.87%, representing a noteworthy 11.49% improvement over the baseline, and the speed remains above 30 frames per second. Additionally, our approach is adaptable to other detection models with different backbones, providing valuable insights for domains such as Urban Air Mobility and autonomous driving. [ABSTRACT FROM AUTHOR]

The novel rotating synthetic aperture (RSA) is a new optical imaging system that uses the method of rotating the rectangular primary mirror for dynamic imaging. It has the advantage of being lightweight, with no need for splicing and real-time surface shape maintenance on orbit. The novel imaging method leads to complex image quality degradation characteristics. Therefore, it is vital to use the image quality improvement method to restore and improve the image quality to meet the application requirements. For the RSA system, a new system that has not been applied in orbit, it is difficult to construct suitable large datasets. Therefore, it is necessary to study and establish the dynamic imaging characteristic model of the RSA system, and on this basis provide data support for the corresponding image super resolution and restoration method through simulation. In this paper, we first analyze the imaging characteristics and mathematically model the rectangular rotary pupil of the RSA system. On this basis, combined with the analysis of the physical interpretation of the blur kernel, we find that the optimal blur kernel is not the point spread function (PSF) of the imaging system. Therefore, the simulation method of convolving the input image directly with the PSF is flawed. Furthermore, the weights of a convolutional neural network (CNN) are the same for each input. This means that the normal convolutional layer is not only difficult to accurately estimate the time-varying blur kernel, but also difficult to adapt to the change in the length–width ratio of the primary mirror. To that end, we propose a blur kernel estimation conditional convolutional neural network (CCNN) that is equivalent to multiple normal CNNs. We extend the CNN to a conditional model by taking an encoding as an additional input and using conditionally parameterized convolutions instead of normal convolutions. The CCNN can simulate the imaging characteristics of the rectangular pupil with different length–width ratios and different rotation angles in a controllable manner. The results of semi-physical experiments show that the proposed simulation method achieves a satisfactory performance, which can provide data and theoretical support for the image restoration and super-resolution method of the RSA system. [ABSTRACT FROM AUTHOR]

Rotating Synthetic Aperture (RSA) technology is one of the distinctly advantageous Earth geostationary orbit optical remote sensing technologies. However, the continuous rotation of the RSA system's rectangular primary mirror results in a discernible drop in resolution along the shorter side of the mirror. Additionally, the captured images exhibit periodic and time-varying characteristics. To improve the image quality to meet interpretation needs, we first delineate the imaging process of the rotating primary mirror and analyze the characteristics of image degradation based on the system's imaging mechanism. Then, we propose a dual super-resolution (SR) framework based on Swin Transformer and introduce a self-supervised learning method for jointly training the unified SR network using wavelet fusion. The self-supervised learning method effectively utilizes the spatiotemporal correlation of the information contained in images captured at different rotation directions of the rectangular pupil. Moreover, the attention mechanism in Transformer can adopt a global perspective and utilize content-based interactions between image content and attention weights to model strong long-range dependencies in remote sensing images. This approach significantly enhances image quality along the pupil's shorter side, consequently yielding superior results. Extensive digital and semi-physical imaging experiments, involving six aspect ratios of the primary mirror, demonstrate that our SR method surpasses state-of-the-art methods. The work in this paper can serve as a valuable reference for future space applications of the RSA technology. • Analyzing the imaging mechanisms of the rotating synthetic aperture system. • Self-supervised learning enhances the utilization of image internal information. • Designing a dual framework that can be trained jointly for image super resolution. • Interactions between image content and attention weights improve performance. • Performance is evaluated through digital and semi-physical experiments. [ABSTRACT FROM AUTHOR]

Due to the complexity of scene interference and the variability of ship scale and position, automatic ship detection in remote sensing images makes for challenging research. The existing deep networks rarely design receptive fields that fit the target scale based on training data. Moreover, most of them ignore the effective retention of position information in the feature extraction process, which reduces the contribution of features to subsequent classification. To overcome these limitations, we propose a novel ship detection framework combining the dilated rate selection and attention-guided feature representation strategies, which can efficiently detect ships of different scales under the interference of complex environments such as clouds, sea clutter and mist. Specifically, we present a dilated convolution parameter search strategy to adaptively select the dilated rate for the multi-branch extraction architecture, adaptively obtaining context information of different receptive fields without sacrificing the image resolution. Moreover, to enhance the spatial position information of the feature maps, we calculate the correlation of spatial points from the vertical and horizontal directions and embed it into the channel compression coding process, thus generating the multi-dimensional feature descriptors which are sensitive to direction and position characteristics of ships. Experimental results on the Airbus dataset demonstrate that the proposed method achieves state-of-the-art performance compared with other detection models. [ABSTRACT FROM AUTHOR]

The YOLO network has been extensively employed in the field of ship detection in optical images. However, the YOLO model rarely considers the global and local relationships in the input image, which limits the final target prediction performance to a certain extent, especially for small ship targets. To address this problem, we propose a novel small ship detection method, which improves the detection accuracy compared with the YOLO-based network architecture and does not increase the amount of computation significantly. Specifically, attention mechanisms in spatial and channel dimensions are proposed to adaptively assign the importance of features in different scales. Moreover, in order to improve the training efficiency and detection accuracy, a new loss function is employed to constrain the detection step, which enables the detector to learn the shape of the ship target more efficiently. The experimental results on a public and high-quality ship dataset indicate that our method realizes state-of-the-art performance in comparison with several widely used advanced approaches. [ABSTRACT FROM AUTHOR]