Search

With the development of remote sensing techniques, a large number of high-resolution aerial images is now available and benefit many applications. Multi-scene recognition plays a key role in applying remote sensing images to these applications, which refers to predicting multiple scenes coexisted in an aerial image and has attracted an increasing attention. Recently, most researchers tend to invent deep learning-based recognition models and has gained great achievements. However, few efforts have been deployed to explaining the success of deep neural networks in multi-scene recognition. To address this, we introduce concept bottleneck model (CBM) to interpreting model performance and propose a novel network, namely Prototype- and Concept-based Interpretable Network (PCINet), that projects aerial imagery into a prototype-concept memory bank and encode their correlations for explaining how a network can identify coexisting scenes in an aerial image. Specifically, the proposed network mainly consists of two branches: prototype matching that measures similarity scores between image features and scene prototypes, and concept bottleneck branches that aligned image features to textual embeddings and compute their relations with concept embeddings. Afterwards, Outputs are integrated for inferring scene categories. Experimental results show that the model enhances interpretability, providing valuable insights for urban planning and resource management, thereby bridging the gap between deep learning models and practical applications. [ABSTRACT FROM AUTHOR]

Structure from Motion (SfM) is a 3D reconstruction framework that has achieved great success on large-scale Unmanned Aerial Vehicle (UAV) images. Due to the high time consumption of feature matching, overlapped match pairs are obtained by image retrieval to improve efficiency. Bag of words (BoW) is commonly used in existing SfM systems. However, the large number of local features and the high dimension of BoW vectors cause image retrieval time-consuming. Recently, the lower dimension learned global features and more efficient feature aggregation methods provide solutions to the problem. Besides, efficient approximate nearest neighbour (ANN) searching can further accelerate image retrieval. Thus, this study conducts an evaluation of image retrieval methods for UAV images in SfM-based reconstruction. First, image retrieval methods with varying combinations of feature descriptors, aggregation strategies, and NN searching algorithms are reviewed and configured for performance evaluation. Second, the selected methods are evaluated in SfM-based reconstruction, in which the image retrieval results are fed into the workflow to guide feature matching and then exploited to create the weighted view graph to achieve parallel SfM reconstruction. Finally, comprehensive tests are conducted to evaluate the performance of selected methods by using three large-scale UAV datasets. The experimental results show that: (1) for feature aggregation and NN searching, Vector of Locally Aggregated Descriptors (VLAD) has superior performance compared with other strategies, and Hierarchical Navigable Small World (HNSW) has better achievement in NNS; (2) among evaluated feature descriptors, with the combination of VLAD and HNSW, the retrieval accuracy of SIFT is still higher than that of the learned local and global features. In a word, the optimal image retrieval method consists of SIFT, VLAD and HNSW, whose retrieval accuracy is higher than BoW by about 2% and efficiency is around 100 times that of BoW. [ABSTRACT FROM AUTHOR]

This study introduces two innovative methods in the research for use in vision-based tunnel inspection vehicles. First, the image-range stitching method is used to map the sequence images acquired by a camera onto a tunnel layout map. This method reduces the tunnel image-stitching problem to the appropriate parameters, thus solving the problem of mapping equations, ranging from camera pixels to the tunnel layout map. The parameters are obtained using a laser scanner. Secondly, traditional label-based deep learning solely perceives the consistency between pixels and semantically labeled samples, making it challenging to effectively address issues with uncertainty and multiplicity. Consequently, we introduce a method that employs a bidirectional heuristic search approach, utilizing randomly generated seed pixels as hints to locate targets that concurrently appear in both the image and the image semantic generation model. The results reveal the potential for cooperation between laser-scanning and camera-imaging technologies and point out a novel approach of crack detection that appears to be more focused on semantic understanding. [ABSTRACT FROM AUTHOR]

Cracks in tunnel linings are the most common tunnel defects. As early indicators of structural deterioration, cracks represent critical problems for the safety of tunnels. Several mobile tunnel inspection systems (MTISs) have been developed for tunnel crack inspection. However, due to the weak signals of cracks, these MTISs require considerable exposure time to capture high-quality tunnel images, necessitating a low travel speed. Meanwhile, traditional crack detection methods encounter difficulties in processing tunnel crack images because of their low contrast and poor continuity. To overcome these challenges, this study presents a new MTIS for fast tunnel crack inspection that consists of a novel mobile imaging module and an automatic crack detection module. The imaging module is composed of an array of high-resolution charge-coupled device (CCD) cameras, a mobile laser scanner, and a lighting array. The core of the crack detection module is a novel lightweight convolutional neural network (CNN) designed for efficient tunnel crack detection, with an effective spatial constraint strategy to guarantee crack continuity. We collected a new tunnel crack dataset consisting of 1,218 images using our mobile imaging module at a driving speed of 80 km/h. Comprehensive experiments were conducted on this dataset to evaluate the performance of our proposed network. The results demonstrate that the presented CNN can effectively detect tunnel cracks with state-of-the-art performance, achieving an F1-score greater than 0.88 and an inference speed of 17 FPS with only 3.4M model parameters. The code and data are available at https://github.com/urban-informatics/LinkCrack. [ABSTRACT FROM AUTHOR]

In this article, we propose a new line segment detection framework based on a novel pixel orientation estimation (POE) method, which detects line segments from binary edge maps and can be combined with any edge detectors. We show its efficiency by testing it in SAR images. The proposed POE-based line segment detection framework aims to leverage the success of deep learning models for edge detection in 1-look SAR images. The novel POE method estimates the orientation of each edge pixel by counting the number of edge pixels along a set of orientations. As the most edge pixels exist along the orientation of the line segment, the orientation of the edge pixel is given by the orientation that gives the maximum number of counts. Counting the number of edge pixels along different orientations is equivalent to convolving the local neighbourhood of each edge pixel with a set of carefully designed window functions with each window function corresponding to a fixed orientation. With the estimated orientations of pixels in the edge map, pixels can be grouped with a region growing step to form line support regions. Regions with their size larger than a size threshold will be accepted. Finally, rectangles are used to approximate the accepted regions and those rectangles are detected line segments. Experiments in both simulated SAR dataset and real SAR images demonstrate the efficiency of the proposed method. In particular, we advance the state-of-the-art performances by 18% (F1-score) on the 1-look dataset simulated from YorkUrban-LineSegment Dataset. [ABSTRACT FROM AUTHOR]

Image super-resolution (SR) methods can generate remote sensing images with high spatial resolution without increasing the cost of acquisition equipment, thereby providing a feasible way to improve the quality of remote sensing images. Clearly, image SR is a severe ill-posed problem. With the development of deep learning, the powerful fitting ability of deep neural networks has solved this problem to some extent. Since the texture information of various remote sensing images are totally different from each other, in this article, we proposed a network based on generative adversarial network (GAN) to achieve high-resolution remote sensing images, named multiattention GAN (MA-GAN). The main body of the generator in MA-GAN contains three blocks: pyramid convolutional residual dense (PCRD) block, attention-based upsampling (AUP) block, and attention-based fusion (AF) block. Specifically, the developed attention pyramid convolutional (AttPConv) operator in the PCRD block combines multiscale convolution and channel attention (CA) to automatically learn and adjust the scale of residuals for better representation. The established AUP block uses pixel attention (PA) to perform arbitrary scales of upsampling. The AF block uses branch attention (BA) to integrate upsampled low-resolution images with high-level features. Besides, the loss function takes both adversarial loss and feature loss into consideration to guide the learning procedure of the generator. We have compared our MA-GAN approach with several state-of-the-art methods on a number of remote sensing scenes, and the experimental results consistently demonstrate the effectiveness of the proposed MA-GAN. For study replication, the source code will be released at: https://github.com/ZhihaoWang1997/MA-GAN. [ABSTRACT FROM AUTHOR]

Visual odometry is the task of estimating the trajectory of the moving agents from consecutive images. It is a hot research topic both in robotic and computer vision communities and facilitates many applications, such as autonomous driving and virtual reality. The conventional odometry methods predict the trajectory by utilizing the multiple view geometry between consecutive overlapping images. However, these methods need to be carefully designed and fine-tuned to work well in different environments. Deep learning has been explored to alleviate the challenge by directly predicting the relative pose from the paired images. Deep learning-based methods usually focus on the consecutive images that are feasible to propagate the error over time. In this paper, graph loss and geodesic rotation loss are proposed to enhance deep learning-based visual odometry methods based on graph constraints and geodesic distance, respectively. The graph loss not only considers the relative pose loss of consecutive images, but also the relative pose of non-consecutive images. The relative pose of non-consecutive images is not directly predicted but computed from the relative pose of consecutive ones. The geodesic rotation loss is constructed by the geodesic distance and the model regresses a Lie algebra so(3) (3D vector). This allows a robust and stable convergence. To increase the efficiency, a random strategy is adopted to select the edges of the graph instead of using all of the edges. This strategy provides additional regularization for training the networks. Extensive experiments are conducted on visual odometry benchmarks, and the obtained results demonstrate that the proposed method has comparable performance to other supervised learning-based methods, as well as monocular camera-based methods. The source code and the weight are made publicly available. [ABSTRACT FROM AUTHOR]

Due to the detailed spectral information through hundreds of narrow spectral bands provided by hyperspectral image (HSI) data, it can be employed to accurately classify diverse materials of interest, which is one of the core applications of hyperspectral remote sensing technology. In recent years, with the rapid development of deep learning, convolutional neural networks (CNNs) have been successfully applied in many fields, including HSI classification. However, the random gradient descent-based parameter updating scheme is too general and leading to the inefficiency of CNN models. Moreover, the high dimensionality and limited training samples of HSI data also exacerbate the overfitting problem. To tackle these issues, in this article, a novel deep network with multilayer and multibranch architecture, named 3-D Gabor CNN (3DG-CNN), is proposed for HSI classification. More precisely, since the predefined 3-D Gabor filters in multiple scales and orientations could well characterize the internal spatial–spectral structure of HSI data from various perspectives, the 3-D Gabor-modulated kernels (3-D GMKs) are employed to replace the random initialization kernels. Moreover, the specially designed multibranch architecture enables the network to better integrating the scalable property of 3-D Gabor filters; thus, the representative ability and robustness of the extracted features can be greatly improved. Alternatively, the number of network parameters is substantially reduced due to the incorporation of 3-D Gabor modulation, relieving the training complexity and also alleviating the training process from overfitting. Experimental results on four real HSI datasets (including two newly released ones in the literature) have demonstrated that the proposed 3DG-CNN model can achieve better performance than several widely used machine-learning-based and deep-learning-based approaches. For the sake of reproducibility, the codes of the proposed 3DG-CNN model are available at http://jiasen.tech/papers/. [ABSTRACT FROM AUTHOR]

Recently, the focus of semantic segmentation research has shifted to the aggregation of context prior and refined boundary. A typical network adopts context aggregation modules to extract rich semantic features. It also utilizes top-down connection and skips connections for refining boundary details. But it still remains disadvantage, an obvious fact is that the problem of false segmentation occurs as the object has very different textures. The fusion of weak semantic and low-level features leads to context prior degradation. To tackle the issue, we propose a simple yet effective network, which integrates dual context prior and spatial propagation-dubbed DSPNet. It extends two mainstreams of current segmentation researches: (1) Designing a dual context prior module, which pays attention to context prior again with a shortcut connection. (2) The network can inherently learn semantic aware affinity values for each pixel and refine the segmentation. We will present detailed comparisons, which perform on PASCAL VOC 2012 and Cityscapes. The result demonstrates the validation of our approach. [ABSTRACT FROM AUTHOR]

In the hyperspectral image, each pixel corresponds to a small area on the Earth’s surface and represents the intrinsic characteristic of objects, which can be applied for recognition of land covers. Nevertheless, hyperspectral image processing should face some critical issues, and a small sample set problem may be the most challenging one in the research. Deep learning (DL), which has successfully been applied in many fields, has also been introduced for hyperspectral image classification. However, the large gap between the massive parameters to be tuned and limited labeled samples can lead to overfitting scenario, inevitably deteriorating the generalization ability of the DL model. In this article, a lightweight convolutional neural network (LWCNN) is proposed for hyperspectral image classification to mainly tackle the small sample set problem. Especially, spatial–spectral Schroedinger eigenmaps (SSSE) feature extraction is first adopted to obtain the joint spatial–spectral information, and the compressed dimensionality could significantly reduce the number of parameters in the following DL model. Second, a dual-scale convolution (DSC) module is carefully designed to address the SSSE features from a 1-D vector viewpoint (the number of parameters is further decreased), and the DSC procedure is successively employed to obtain the hierarchical structure description that could represent data distribution from different aspects. Subsequently, the feature vectors from all DSC layers are separately filtered by a new bichannel fusion (BCF) module, which could well encode both the intrinsic and contextual information inside DSC features. Finally, the filtered features are concatenated together and imported into a global average pooling classifier to achieve the predicted probability of each category. Experimental results on three famous hyperspectral image data sets illustrate that the developed LWCNN approach is advantageous in both the efficiency and robustness sides for hyperspectral image classification tasks and outperforms other state-of-the-art methods (both traditional-based and DL-based) with very limited labeled samples. [ABSTRACT FROM AUTHOR]

With the rapid growing of remotely sensed imagery data, there is a high demand for effective and efficient image retrieval tools to manage and exploit such data. In this letter, we present a novel content-based remote sensing image retrieval (RSIR) method based on Triplet deep metric learning convolutional neural network (CNN). By constructing a Triplet network with metric learning objective function, we extract the representative features of the images in a semantic space in which images from the same class are close to each other while those from different classes are far apart. In such a semantic space, simple metric measures such as Euclidean distance can be used directly to compare the similarity of images and effectively retrieve images of the same class. We also investigate a supervised and an unsupervised learning methods for reducing the dimensionality of the learned semantic features. We present comprehensive experimental results on two public RSIR datasets and show that our method significantly outperforms state-of-the-art. [ABSTRACT FROM AUTHOR]

The presence of clouds greatly reduces the ground information of high-resolution satellite data. In order to improve the utilization of high-resolution satellite data, this article presents a cloud removal method based on deep learning. This is the first end-to-end architecture that has great potential to detect and remove clouds from high-resolution satellite data. For cloud detection, a convolution neural network (CNN) architecture is used to detect them. For cloud removal, the content generation network, the texture generation network, and the spectrum generation network based on traditional CNN are proposed. The proposed CNN architecture can use multisource data (content, texture, and spectral) as an input of the unified framework. The results of both the simulated and real image experiments demonstrate that the proposed method is robust and can effectively remove thick clouds, thin clouds, and cloud shadows. In addition, compared with some existing methods, the proposed method can recover land cover information accurately. [ABSTRACT FROM AUTHOR]

In the indoor environment, the activity of the pedestrian can reflect some semantic information. These activities can be used as the landmarks for indoor localization. In this paper, we propose a pedestrian activities recognition method based on a convolutional neural network. A new convolutional neural network has been designed to learn the proper features automatically. Experiments show that the proposed method achieves approximately 98% accuracy in about 2 s in identifying nine types of activities, including still, walk, upstairs, up elevator, up escalator, down elevator, down escalator, downstairs and turning. Moreover, we have built a pedestrian activity database, which contains more than 6 GB of data of accelerometers, magnetometers, gyroscopes and barometers collected with various types of smartphones. We will make it public to contribute to academic research. [ABSTRACT FROM AUTHOR]

Urban land use is key to rational urban planning and management. Traditional land use classification methods rely heavily on domain experts, which is both expensive and inefficient. In this paper, deep neural network-based approaches are presented to label urban land use at pixel level using high-resolution aerial images and ground-level street view images. We use a deep neural network to extract semantic features from sparsely distributed street view images and interpolate them in the spatial domain to match the spatial resolution of the aerial images, which are then fused together through a deep neural network for classifying land use categories. Our methods are tested on a large publicly available aerial and street view images dataset of New York City, and the results show that using aerial images alone can achieve relatively high classification accuracy, the ground-level street view images contain useful information for urban land use classification, and fusing street image features with aerial images can improve classification accuracy. Moreover, we present experimental studies to show that street view images add more values when the resolutions of the aerial images are lower, and we also present case studies to illustrate how street view images provide useful auxiliary information to aerial images to boost performances. [ABSTRACT FROM AUTHOR]

Camera-based indoor localization is a fundamental aspect of indoor navigation, virtual reality, and location-based services. Deep learning methods have exhibited remarkable performance with low storage requirements and high efficiency. However, existing methods mainly derive features implicitly for pose regression without considering explicit structure information from images. This paper proposes that incorporating such information can improve the localization performance of learning-based approaches. We extract structure information from RGB images in the form of depth maps and edge maps and design two modules for depth-weighted and edge-weighted feature fusion. These modules are integrated into the pose regression network to enhance pose prediction. Furthermore, we employ a self-attention module for high-level feature extraction to augment the network capacity. Extensive experiments are conducted on the publicly available 7Scenes and 12Scenes datasets, and the results demonstrate that the proposed method achieves high localization performance, with an average positional error of 0. 19 m and 0. 16 m , respectively. The code for this work is available at https://github.com/lqing900205/structureLoc. [ABSTRACT FROM AUTHOR]

Image-based indoor localization provides fundamental support for applications such as indoor navigation, virtual reality, and location-based services. Most research focuses on developing methods in good lighting conditions via RGB images; while for low lighting situations, especially at night, RGB-based methods cannot perform well. Depth images are promising alternative in such conditions as they record geometrical information instead of texture information, making it possible to work in low lighting scenarios. Current depth image-based methods, either retrieval-based methods or 3D registration-based methods, are inefficient due to its high computation overhead, preventing the wide applications. To address this issue, we propose a fast coarse-to-fine localization framework for dark environment via deep learning and keypoint-based geometry alignment. In the coarse localization phase, we jointly perform the depth completion and pose regression to relieve the occlusion caused appearance variance in depth images. In the refinement phase, keypoints are used instead of whole depth image points under the ICP alignment framework to increase the localization efficiency. The keypoints are detected on depth feature maps weakly supervised with pose regression. The experiments on the open available 7Scenes dataset show that the proposed method obtain positional accuracy of 0.143 m and orientational accuracy of 5.275°in average and only cost 0.8s for a single depth image. The code for the proposed work is available at https://github.com/lqing900205/KeyPointDepthLocalization [ABSTRACT FROM AUTHOR]

Ancient paintings are valuable for historians and archeologists to study the humanities, customs and economy of the corresponding eras. For this purpose, it is important to first determine the era in which a painting was drawn. This problem can be very challenging when the paintings from different eras present a same topic and only show subtle difference in terms of the painting styles. In this paper, we propose a novel computational approach to address this problem by using the appearance and shape features extracted from the paintings. In this approach, we first extract the appearance and shape features using the SIFT and k AS descriptors, respectively. We then encode these features with deep learning in an unsupervised way. Finally, we combine all the features in the form of bag-of-visual-words and train a classifier in a supervised fashion. In the experiments, we collect 660 Flying-Apsaras paintings from Mogao Grottoes in Dunhuang, China and classify them into three different eras, with very promising results. [ABSTRACT FROM AUTHOR]

Although the deep learning-based indoor image localization has made significant improvement in terms of accuracy, efficiency, and storage requirement of large indoor scenes, the need for collecting huge labeled training data severely limits its practical application. Recently, the synthetic images rendered from widely available 3D models have shown promising potential to relieve the data collection problem. However, due to the dramatic differences between the synthetic and real images, the localization accuracy of approaches trained on synthetic images is not comparable to the methods trained on real images. In this paper, we propose a domain adaptation-based approach to address this issue. Specifically, the proposed approach mainly contains a model consisting of a multi-level constrained pose regression network and a feature-level discriminator network. The discriminator network forces the pose regression network to generate domain-invariant features from real and synthetic images by adversarial learning and thus reduces the performance gaps. In addition, the multi-level constraints further enhance the localization accuracy of pose regression. We perform extensive experiments on open-source rendering images in different settings. The results show that the proposed method significantly improves the performance. The code for the proposed work is available at https://github.com/lqing900205/BIM_domainadaptation. [ABSTRACT FROM AUTHOR]

6DOF camera relocalization is an important component of autonomous driving and navigation. Deep learning has recently emerged as a promising technique to tackle this problem. In this paper, we present a novel relative geometry-aware Siamese neural network to enhance the performance of deep learning-based methods through explicitly exploiting the relative geometry constraints between images. We perform multi-task learning and predict the absolute and relative poses simultaneously. We regularize the shared-weight twin networks in both the pose and feature domains to ensure that the estimated poses are globally as well as locally correct. We employ metric learning and design a novel adaptive metric distance loss to learn a feature that is capable of distinguishing poses of visually similar images from different locations.We evaluate the proposed method on public indoor and outdoor benchmarks and the experimental results demonstrate that our method can significantly improve localization performance. Furthermore, extensive ablation evaluations are conducted to demonstrate the effectiveness of different terms of the loss function. [ABSTRACT FROM AUTHOR]

Urban region function recognition is key to rational urban planning and management. Due to the complex socioeconomic nature of functional land use, recognizing urban region function in high-density cities using remote sensing images alone is difficult. The inclusion of social sensing has the potential to improve the function classification performance. However, effectively integrating the multi-source and multi-modal remote and social sensing data remains technically challenging. In this paper, we have proposed a novel end-to-end deep learning-based remote and social sensing data fusion model to address this issue. Two neural network based methods, one based on a 1-dimensional convolutional neural network (CNN) and the other based on a long short-term memory (LSTM) network, have been developed to automatically extract discriminative time-dependent social sensing signature features, which are fused with remote sensing image features extracted via a residual neural network. One of the major difficulties in exploiting social and remote sensing data is that the two data sources are asynchronous. We have developed a deep learning-based strategy to address this missing modality problem by enforcing cross-modal feature consistency (CMFC) and cross-modal triplet (CMT) constraints. We train the model in an end-to-end manner by simultaneously optimizing three costs, including the classification cost, the CMFC cost and the CMT cost. Extensive experiments have been conducted on publicly available datasets to demonstrate the effectiveness of the proposed method in fusing remote and social sensing data for urban region function recognition. The results show that the seemingly unrelated physically sensed image data and social activities sensed signatures can indeed complement each other to help enhance the accuracy of urban region function recognition. [ABSTRACT FROM AUTHOR]

Abstract The outstanding accuracy and spatial resolution of airborne light detection and ranging (LiDAR) systems allow for very detailed urban monitoring. Classification is a crucial step in LiDAR data processing, as many applications, e.g., 3D city modeling, building extraction, and digital elevation model (DEM) generation, rely on classified results. In this study, we present a novel LiDAR classification approach that uses simple pixel comparison features instead of the manually designed features used in many previous studies. The proposed features are generated by the computed height difference between two randomly selected neighboring pixels. In this way, the feature design does not require prior knowledge or human effort. More importantly, the features encode contextual information and are extremely quick to compute. We apply a random forest classifier to these features and a majority analysis postprocessing step to refine the classification results. The experiments undertaken in this study achieved an overall accuracy of 87.2%, which can be considered good given that only height information from the LiDAR data was used. The results were better than those obtained by replacing the proposed features with five widely accepted man-made features. We conducted algorithm parameter setting tests and an importance analysis to explore how the algorithm works. We found that the pixel pairs directing along the object structure and with a distance of the approximate object size can generate more discriminative pixel comparison features. Comparison with other benchmark results shows that this algorithm can approach the performance of state-of-the-art deep learning algorithms and exceed them in computational efficiency. We conclude that the proposed algorithm has high potential for urban LiDAR classification. [ABSTRACT FROM AUTHOR]

Inspired by heterogeneity of rapid-increasing 3D pavement data and the generalization ability of transfer learning, a robust and generalized framework for cross-scene 3D pavement-crack detection and attribute extraction was proposed in this paper, called profile component decomposition model with holistically nested edge detection (PCDM-HED). The core purpose of the PCDM-HED is to construct the enhanced deep edge features. By applying profile frequency, sparse characteristics of 3D profiles, and fusing the multiscale and multilevel edge characteristics of 3D depth maps in HED network, the robust enhanced feature can highlight the essential properties of cracks in heterogeneous 3D data. It overcomes the complex domain shifts caused by different 3D imaging conditions, pavement textures, and crack distributions in heterogeneous data. Cross-domain transfer experiments were carried out over seven 3D/2D datasets with 915 pavement sections. The results show that proposed PCDM-HED achieved average buffered Hausdorff scores of 90.17 to 96.42, recall scores of 0.84 to 0.91, and F-values of 0.85 to 0.89 in six different datasets without labeled samples. Compared with 9 groups of comparison results, including the traditional and related state-of-the-art methods, the transfer generalization effect of proposed PCDM-HED is more than 23% higher than that of comparison results. The proposed PCDM-HED makes full use of limited off-the-shelf samples, demonstrated strong transfer learning capability. It provides an effective solution for heterogeneous 3D pavement-crack detection tasks in engineering, in the case of limited labeled samples or even no corresponding labeled samples. • A 3D pavement crack detection with generalized transfer learning ability is proposed. • An enhanced deep edge feature is developed for heterogeneous 3D pavement data. • The PCDM-HED achieves BHS of 90.17 to 96.42 in heterogeneous without labeled samples. • The PCDM-HED demonstrates generalization ability across different 3D pavement scenes. [ABSTRACT FROM AUTHOR]

As an important task for the management of bike sharing systems, accurate forecast of travel demand could facilitate dispatch and relocation of bicycles to improve user satisfaction. In recent years, many deep learning algorithms have been introduced to improve bicycle usage forecast. A typical practice is to integrate convolutional (CNN) and recurrent neural network (RNN) to capture spatial–temporal dependency in historical travel demand. For typical CNN, the convolution operation is conducted through a kernel that moves across a "matrix-format" city to extract features over spatially adjacent urban areas. This practice assumes that areas close to each other could provide useful information that improves prediction accuracy. However, bicycle usage in neighboring areas might not always be similar, given spatial variations in built environment characteristics and travel behavior that affect cycling activities. Yet, areas that are far apart can be relatively more similar in temporal usage patterns. To utilize the hidden linkage among these distant urban areas, the study proposes an irregular convolutional Long-Short Term Memory model (IrConv+LSTM) to improve short-term bike sharing demand forecast. The model modifies traditional CNN with irregular convolutional architecture to leverage the hidden linkage among "semantic neighbors". The proposed model is evaluated with a set of benchmark models in five study sites, which include one dockless bike sharing system in Singapore, and four station-based systems in Chicago, Washington, D.C., New York, and London. We find that IrConv+LSTM outperforms other benchmark models in the five cities. The model also achieves superior performance in areas with varying levels of bicycle usage and during peak periods. The findings suggest that "thinking beyond spatial neighbors" can further improve short-term travel demand prediction of urban bike sharing systems. • Propose an Irregular Convolutional Long Short-Term Memory Network (IrConv+LSTM) model. • Introduce the novel concept of semantic neighbors. • IrConv+LSTM with semantic neighbors outperforms benchmark models in the five cities. • Leveraging hidden linkage over areas with similar usage patterns facilitates the demand forecast. [ABSTRACT FROM AUTHOR]

Most automated sewer inspection tasks are based on closed-circuit television (CCTV) methods and focus on image classification or object detection but fail to obtain information on fine-grained sewer defects. Targeting the sewer defect detection from a video and performing the sewer inspection in a lightweight, low-cost, and practical manner, this study develops a sewer detection framework based on computer vision technology for the sewer floating capsule robot. The proposed framework performs three main sub-tasks, including instance segmentation, real-time sewer inspection device localization, and real three-dimensional (3D) model reconstruction, to realize sewer defect detection. An improved mask regional convolutional neural network (Mask RCNN), which integrates split attention module and balanced L1 loss module, is proposed for robust feature representation. In addition, an improved data augmentation method developed according to the sewer defect instance segmentation tasks is introduced. The 3D reconstruction and real-time localization of a sewer scene are achieved using the structure-from-motion techniques for the sewer floating capsule robot. Extensive experiments are conducted, and the experimental results show that the proposed method is effective and robust. The average precision of instance segmentation at the intersection of the union value of 0.5 is 92.7%, and the maximum 3D model measurement error is 1 m. However, video sequence information and multi-sensor fusion, combining the inertial measurement unit and vision technique, could be studied in the future to achieve better generalization and robust results. • A low-cost sewer inspecting robot is developed. • Split attention mechanism is integrated into the model's backbone. • Balanced L1 loss is integrated into the loss module. • Data augmentation is performed for sewer instance segmentation. • Sewer image and device localization is achieved, and the 3D model is constructed. [ABSTRACT FROM AUTHOR]

Surface water mapping is very important for studying its role in global water cycle, flooding dynamic monitoring, and water resources management. The most famous techniques for water body extraction like those based on water spectral indices (WSI) require rich spectral information. However, the WSI methods are no longer practical in high-resolution multispectral (MS) images due to insufficient spectral information. In addition, surface water mapping faces an utmost overestimation issue because shadows are misclassified as water bodies. To address the above-mentioned problems, in this paper, a novel refined water body extraction neural network (WBE-NN) is proposed. The global spatial-spectral convolution (GSSC) module is developed to enhance surface water body features. A novel multiscale learning module is designed to extract multi-scale contextual information. In addition, the surface water body boundary refinement (SWBBR) module is adopted to enhance surface water body boundaries. The results show that the proposed method achieved good performance with a mean overall accuracy of 98.97%, a mean Kappa coefficient of 94.78%, and a mean boundary overall accuracy of 98.01%. Therefore, WBE-NN can be used for mapping surface water with high accuracy in complex areas. [ABSTRACT FROM AUTHOR]