Search

Prediction of Remaining Useful Life (RUL) for Rolling Element Bearings (REB) has attracted widespread attention from academia and industry. However, there are still several bottlenecks, including the effective utilization of multi-sensor data, the interpretability of prediction models, and the prediction across the entire life cycle, which limit prediction accuracy. In view of that, we propose a knowledge-based explainable life-cycle RUL prediction framework. First, considering the feature fusion of fast-changing signals, the Pearson correlation coefficient matrix and feature transformation objective function are incorporated to an Improved Graph Convolutional Autoencoder. Furthermore, to integrate the multi-source signals, a Cascaded Multi-head Self-attention Autoencoder with Characteristic Guidance is proposed to construct health indicators. Then, the whole life cycle of REB is divided into different stages based on the Continuous Gradient Recognition with Outlier Detection. With the development of Measurement-based Correction Life Formula and Bidirectional Recursive Gated Dual Attention Unit, accurate life-cycle RUL prediction is achieved. Data from self-designed test rig and PHM 2012 Prognostic challenge datasets are analyzed with the proposed framework and five existing prediction models. Compared with the strongest prediction model among the five, the proposed framework demonstrates significant improvements. For the data from self-designed test rig, there is a 1.66 % enhancement in Corrected Cumulative Relative Accuracy (CCRA) and a 49.00 % improvement in Coefficient of Determination (R2). For the PHM 2012 datasets, there is a 4.04 % increase in CCRA and a 120.72 % boost in R2. [Display omitted] • Advocate explainable knowledge-data integration for RUL prediction throughout the life cycle, leveraging multi-sensor data. • Introduce a novel unsupervised learning framework that integrates IGCA and CMSACG to construct HI. • A two-stage prediction framework, fusing MCLF and BR-GDAU, is proposed for predicting the life cycle RUL. • The effectiveness of the proposed framework is validated using both Self-Designed Experiment-Derived and PHM 2012 Prognostic Challenge Bearing Datasets. [ABSTRACT FROM AUTHOR]

Recommendation systems research is a subfield of information retrieval, as these systems recommend appropriate items to users during their visits. Appropriate recommendation results will help users save time searching while increasing productivity at work, travel, or shopping. The problem becomes more difficult when the items are geographical locations on the ground, as they are associated with a wealth of contextual information, such as geographical location, opening time, and sequence of related locations. Furthermore, on social networking platforms that allow users to check in or express interest when visiting a specific location, their friends receive this signal by spreading the word on that online social network. Consideration should be given to relationship data extracted from online social networking platforms, as well as their impact on the geolocation recommendation process. In this study, we compare the similarity of geographic locations based on their distance on the ground and their correlation with users who have checked in at those locations. When calculating feature embeddings for users and locations, social relationships are also considered as attention signals. The similarity value between location and correlation between users will be exploited in the overall architecture of the recommendation model, which will employ graph convolution networks to generate recommendations with high precision and recall. The proposed model is implemented and executed on popular datasets, then compared to baseline models to assess its overall effectiveness. [ABSTRACT FROM AUTHOR]

In the domain of 5G network management, accurately predicting traffic volumes at base stations remains a critical yet challenging endeavor, primarily due to the complexities inherent in the spatial and temporal data interactions. Current methods often fall short in effectively harnessing long-term trends and spatial interconnections among base stations. To bridge these gaps, this paper introduces the GCformer model, a novel approach that capitalizes on both spatial relationships and temporal patterns for multi-base station traffic prediction. Spatially, the proposed model employs graph convolutional networks to integrate diverse spatial information and construct insightful adjacency matrices that includes Euclidean distances and non-Euclidean distances (area types of base station locations and similarities in traffic flow among various stations), thereby enhancing the predictability of traffic dynamics. Temporally, the application of the Transformer's attention mechanism enables better capture of long-term relational dependencies in the temporal domain of 5 G base station traffic data. Additionally, a time-variant optimization module is designed to establish diurnal cycle data for each base station's traffic, replacing the traditional positional encoding with a more nuanced model that improves the learning of historical data correlations. Empirical results from exhaustive case studies confirm the superiority of the GCformer model in forecasting traffic volumes. The GCformer exhibits a 4.01 % improvement in mean squared error and a 3.37 % enhancement in mean absolute error compared to the best-performing baseline model, showcasing its potential to significantly improve operational strategies in 5G networks. • GCformer, combing graph convolution model and Transformer model, for 5G base station traffic volume prediction is proposed. • Multi-graphs viewed as different spatial relationships are proposed into the spatial learning. • Attention-based mechanism combing Time Variant Optimization Module (TVOM) are integrated into the temporal learning. [ABSTRACT FROM AUTHOR]

Graph Convolution Networks (GCNs) have been widely used in skeleton-based action recognition. Although there are significant progress, the inherent limitation still lies in the restricted receptive field of GCN, hindering its ability to extract global dependencies effectively. And the joints that are structurally separated can also have strong correlation. Previous works rarely explore local and global correlations of joints, leading to insufficiently model the complex dynamics of skeleton sequences. To address this issue, we propose a GCN and Transformer complementary network (GTC-Net) that allows parallel communications between GCN and Transformer domains. Specifically, we introduce a graph convolution and self-attention combined module (GAM), which can effectively leverage the complementarity of GCN and self-attention to perceive local and global dependencies of joints for the human body. Furthermore, in order to address the problems of long-term sequence ordering and position detection, we design a position-aware module (PAM), which can explicitly capture the ordering information and unique identity information for body joints of skeleton sequence. Extensive experiments on NTU RGB+D 60 and NTU RGB+D 120 datasets are conducted to evaluate our proposed method. The results demonstrate that our method can achieve competitive results on both datasets. • We propose a combined GCN and self-attention module for parallel communication and effective correlation modeling. • We design a position-aware module to handle long-term sequence ordering and position detection. • Experiments on NTU RGBD+60 and NTU RGBD+120 demonstrates competitive results. [ABSTRACT FROM AUTHOR]

Temporal action segmentation is a task for understanding human activities in long-term videos. Most of the efforts have been focused on dense-frame action, which relies on strong correlations between frames. However, in the figure skating scene, technical actions are sparsely shown in the video. This brings new challenges: a large amount of redundant temporal information leads to weak frame correlation. To end this, we propose a Bidirectional Temporal and Frame-Segment Attention Module (FSAM). Specifically, we propose an additional reverse-temporal input stream to enhance frame correlation, learned by fusing bidirectional temporal features. In addition, the proposed FSAM contains a Multi-stage segment-aware GCN and decoder interaction module, aiming to learn the correlation between segment features across time domains and integrate embeddings between frame and segment representations. To evaluate our approach, we propose the Figure Skating Sparse Action Segmentation (FSSAS) dataset: The dataset comprises 100 samples of the Olympic figure skating final and semi-final competition, with more than 50 different men and women athletes. Extensive experiments show that our method achieves an accuracy of 87.75 and an edit score of 90.18 on the FSSAS dataset. • We propose a bidirectional temporal framework, which improves inter-frame coherence. • We propose a frame-segment attention module to learn the correlation of segments. • Our method achieves SOTA results on the sparse action dataset of figure skating. [ABSTRACT FROM AUTHOR]

The proliferation of portable electroencephalogram (EEG) recording devices has made it practically feasible to develop the motor imagery (MI) based brain–computer interfaces (BCIs). However, the low signal-to-noise ratio of EEG signals for abstract MI tasks, limited data, limited EEG channels, and strong inter- and intra-subject variability pose significant challenges for MI-task recognition. This paper proposes a transfer learning assisted graph convolutional network (GCN) modeling approach for cross-dataset MI decoding, one of the most challenging issues in this field. In the experiments, a multi-channel dataset with 62 electrodes and a few-channel dataset with 8 electrodes are utilized for cross-dataset modeling. To harness multi-channel information, we utilize the GCN module to aggregate topological features. The pre-trained model is guided with few-channel signals as inputs through a knowledge distillation framework. Subsequently, the pre-trained model is adapted to the few-channel dataset using a transfer learning strategy with minimal data training. Experiment results show that the proposed model achieves 3.92% and 3.83% more accuracy improvement compared with state-of-the-art models in the cross-validation and cross-session scenario respectively, demonstrating the effectiveness of the proposed approach in cross-dataset MI-EEG decoding, thus enabling more effective MI-BCI applications. [ABSTRACT FROM AUTHOR]

Deepfake videos are increasingly used in spreading fake news or propaganda having a serious impact on people and society. Traditional deepfake detectors exploit spatial and/or temporal inconsistencies to differentiate between real and fake videos. Owing to the rapidly advancing deepfake creation algorithms, the latest detectors have made use of physiological and biological facial features to create more generic solutions. Our proposed solution uses facial landmarks as the physiological identifiers of a person's face and through them develops a relationship between facial areas in normal speech and tampered speech. By creating a graph structure from the resulting sparse data, we were able to use a spatio-temporal graph convolutional network for classification, which has significantly fewer parameters and a shorter training time than traditional CNNs. We conducted a multitude of experiments on 3 datasets, utilizing spatio-temporal features. The results demonstrate that this technique has better generalization, and high performance compared to latest research in deepfake detection without the reliance on large deep learning models which are tuned to learning image discrepancies more than data patterns. Moreover, our use of facial landmark-based features with a graph structure paves the way for the development of an explainable AI model that can be relied on. • Deepfake video generation algorithms modify the data associated with facial landmarks to create fake images and videos which are used as a source for spreading fake news. • Facial landmarks represent an important structural representation of the face. • A feature vector can be found on each facial landmark to create a representation of the facial structure. • Facial structural data in form of landmarks and feature vectors can be classified using Graph CNNs to detect deepfakes since GCNs are better suited for sparse geometric data. [ABSTRACT FROM AUTHOR]

[Display omitted] • Employ contour features as geometric priors to preserve the outline details of lesions and explicitly model the correlation between region and contour. • Propose a parallel graph reasoning strategy to model the long-range relationship between adjacent pixels and analyze the global dependencies along the channel dimension of feature maps. • The improvements of the proposed method compare to the state-of-the-art models are validated on two public LiTS17 and 3DIRCADb datasets. The accurate detection and segmentation of liver cancers from abdominal CT scans is critical. However, segmenting liver tumors presents significant hurdles due to indistinct lesion boundaries and ignoring the correlation between target and outlines. In this paper, we propose the Parallel Graph Convolutional Network (PGC-Net), a completely novel segmentation framework for liver tumors. With regard to segmentation against constraints, we specifically use contour-induced parallel graph reasoning for quick yet efficient segmentation. First, we use a Pyramid Vision Transformer that has already been trained to extract multi-scale features of region and contour. In order to project the pixels into two distinct high-dimensional areas, we secondly use the parallel graph reasoning strategy, where the vertices are weighted in accordance with the geometric prior of the contour. Through the process of graph convolution, the complementary properties of region and contour also propagate the information. Finally, we project back to the original pixel space for the prediction using the refined features deduced from the graph. Experimental results on two available datasets, LiTS17 (with an average Dice score of 73.63%) and 3DIRCADb (with an average Dice score of 74.16%). Our framework focused on the interaction between two orthogonal graphs and contour information, which has the potential to improve the accuracy and efficiency of liver tumor segmentation. [ABSTRACT FROM AUTHOR]

• We proposed a bi-channel convolution framework to estimate the CBT end-to-end; • We improved topological similarity constraints between the CBT and multi-view networks; • We constructed harmonic bases with the CBT to explore the alterations in anatomical features in the harmonic domain. Multi-view networks constructed from multiple brain functional or structural measures (e.g., fractional anisotropy, fiber number, and fiber length, etc.) describe brain connectivity from different views. The connectional brain template (CBT) of multi-view networks from a given population provides a standard graph template incorporating complementary information for analyzing intergroup differences. However, preserving the complex topology of multi-view networks and building the CBT end-to-end remains challenging. Hence, we proposed a bi-channel convolution framework, including a local connection channel (LCC) with edge-conditioned convolution layers and a global network channel (GNC) with graph convolution network, to estimate the CBT in an end-to-end manner. The LCC captured the local connection topology across individual networks, and the GNC learned the global network topology through the high-order population network built by the similarity between the node strength distributions of individual networks for each view. Additionally, we improved topological similarity constraint in the loss function by minimizing the KL divergence of the node strength and degree distributions between the CBT and multi-view networks. Compared with four existing methods, our CBT exhibited optimal centrality, topological structure retention ability and differentiation between the younger and older age groups. Using the harmonic bases of the CBT, we found that the total harmonic energy of cortical thickness presented significant differences among different age groups. Our method provided a standard common template for examining the structural connection changes in the spatial domain and the anatomical features alterations in the harmonic domain, thus distinguishing the different population groups. [ABSTRACT FROM AUTHOR]

This paper deals with the problem of content-based image retrieval (CBIR) of very high resolution (VHR) remote sensing (RS) images using the notion of a novel Siamese graph convolution network (SGCN). The GCN model has recently gained popularity in learning representations for irregular domain data including graphs. In the same line, we argue the effectiveness of region adjacency graph (RAG) based image representations for VHR RS scenes in terms of localized regions. This technique captures important scene information which can further aid in a better image to image correspondence. However, standard GCN features, in general, lacks discriminative property for fine-grained classes. These features may not be optimal for the task of CBIR in many cases with coherent local characteristics. As a remedy, we propose the SGCN architecture for assessing the similarity between a pair of graphs which can be trained with the contrastive loss function. Given the RAG representations, the aim is to learn an embedding space that pulls semantically coherent images closer while pushing dissimilar samples far apart. In order to ensure a quick response while performing the retrieval using a given similarity measure, the embedding space is kept constrained. We implement the proposed embeddings for the task of CBIR for RS data on the popular UC-Merced dataset and the PatternNet dataset where improved performance can be observed. • Very high resolution (VHR) remote sensing image retrieval. • Novel Siamese graph Convolutional network architecture which is end to end to trainable. • Improved retrieval performance using the embeddings from the Siamese graph convnet. • Extensive experimentations on the benchmark UC-Merced and large-scale PatternNet datasets. [ABSTRACT FROM AUTHOR]

Sleep is crucial for human health. Automatic sleep stage classification based on polysomnography (PSG) is meaningful for the diagnosis of sleep diseases, which has attracted extensive attention recently. Most existed works learn the representations of PSG based on the conventional grid data structure, ignoring the topological channel correlations among different body regions in non-Euclidean space. To this end, we propose a deep learning model named SAGSleepNet, which transforms PSG into a robust graph structure by neural networks and self-attention mechanism, utilizing graph convolution network (GCN) and bidirectional gated recurrent unit (BiGRU) to capture epoch-level topological channel correlations and sequence-level temporal sleep transitions of PSG, respectively. The final learned representation is fed into a softmax layer to train an end-to-end model for automatic sleep staging. Experimental results on three public datasets (i.e., SleepEDFx, UCD, CAP) show that SAGSleepNet obtains the best classification performance compared with several baselines, including accuracy of 0.807 ,0.765, and 0.746, F1-score of 0.744, 0.750, and 0.695, and Cohen's Kappa (κ) of 0.721, 0.690, and 0.660. In general, our work provides a reasonable graph structure to model PSG epoch, and makes contribution to explore implicit channel-wise information of PSG by deep learning techniques. [Display omitted] • Automatic sleep staging based on polysomnography is meaningful for sleep medicine. • Topological channel correlations of polysomnography is close to body correlations. • Transforming polysomnography to robust graph structure by self-attention mechanism. • Graph convolution network effectively learns the topological information of graph. • The proposed hierarchical deep neural networks improves automatic sleep staging. [ABSTRACT FROM AUTHOR]

Accurate flight ground service time prediction plays a key role in the efficient operation of airports, which can help managers make better resource scheduling optimization decisions, reduce flight delays and improve airport operation capabilities. However, since flight ground service data is typically acquired by sensor equipment or manually recorded, data missing and abnormalities caused by sensor failure or manual recording errors are an unavoidable occurrence, making the prediction problem challenging. Although missing values can be imputed, the traditional methods usually fill in missing values and then make predictions separately, which leads to a two-step process and increases the prediction error. Besides, the existing one-step methods still suffer some limitations and shortcomings: failure to integrate bidirectional spatial–temporal dependencies leads to unsatisfying model prediction accuracy and poor robustness under high data missing rates, which cannot be directly used in the study of flight ground service time prediction with missing values. To solve this problem, we presented a sliding bidirectional graph convention network model framework that capable of dealing predicting with missing values. First, we proposed a new adjacency matrix construction method to capture more comprehensive spatial–temporal dependencies and developed a graph convention network based missing value imputation unity (GCNM) to inference and filling missing features. Then, multiple GCNM units are stacked to develop a bidirectional module BDGCNM to realize the filling and prediction into one step. Finally, we introduced a variable sliding window mechanism to improve the model's robustness. Extensive experiments on real airport data sets and PEMS traffic data sets show that our method is more accurate and robust than the current state-of-the-art. • A sliding bidirectional graph convention network model framework is presented for airport flight ground service time prediction with missing values, which integrating the history missing data imputation and future prediction into a onestep process. • A new adjacency matrix construction method is developed to capture more comprehensive spatial–temporal dependencies features. • We stack multiple GCNM units form bidirectional directions to form the BDGCNM module, which improves model prediction accuracy when compared to existing recurrent neural network-based methods. • The prediction robustness increases when sliding mechanism are applied on high missing rate data, and experimental results on three real-world data sets show that the proposed model achieves an overall performance improvement. [ABSTRACT FROM AUTHOR]

In existing motion prediction methods that use graph convolution networks, motion sequences are transformed to a spectral domain, and future motions are predicted through graph spectral filtering for the transformed spectral sequences. However, because the conventional spectral transform method uses a predetermined spectral basis, the motion prediction does not work well for aperiodic or complicated motions. To overcome this problem, we propose a method to learn spectral domain transforms from motion sequences in the training dataset. To this end, two methods are attempted: one for learning the frequency of each spectral basis, and another for learning the values of the basis function directly. Through experiments on representative 3D human motion benchmarks, H3.6M and CMU Mocap, we demonstrate that both of the proposed methods consistently outperform the baseline method. In particular, the method of directly learning the basis function outperforms the state-of-the-art methods. We also demonstrate that the proposed method yields realistic predictions, even for aperiodic and complicated action categories. • Proposes a novel GCN-based motion prediction method of learning spectral transform. • Tackles the limitations of fixed spectral basis functions in conventional methods. • Automatically learns spectral basis without manual setting of the number of basis. • Generates realistic prediction results for aperiodic and dynamic action categories. • Sets the new best performance in 3D human motion prediction on H3.6M and CMU Mocap. [ABSTRACT FROM AUTHOR]

Real-time and accurate Tropical Cyclone (TC) precipitation nowcasting plays an important role in disaster prevention and mitigation. The strong ability of deep learning has been proved in many precipitation nowcasting studies. However, few deep learning solutions focus on the precipitation caused by TC, which has complex patterns and is potential for disasters. In this paper, we develop a novel framework for TC precipitation nowcasting. To effectively utilize the TC information, we design a multi-source information fusion mechanism between precipitation and TC. More importantly, we propose a Graph-guided Spatio-Temporal Module (GSTM), which designs a construction scheme of dynamic reasoning graphs to capture the spatio-temporal relationship between intra-frame and inter-frame. It is also the first attempt to introduce the graph concept into precipitation nowcasting. We utilize the TC and precipitation data from 2017 to 2019 in the Northwest Pacific Ocean and divide the data into training (3524 sequences), validation (352 sequences), and test set (705 sequences) at a ratio of 10:1:2. Extensive experiments demonstrate that compared with the Convolution LSTM model, our model obtains a significant improvement by 2.74%, 4.24%, 6.29%, 5.78% in F1-score when the precipitation intensity is 5, 10, 20, 30 mm/h, respectively. Meanwhile, we select three typical TC precipitation cases for visual analysis, which proves that our model has excellent perception ability in the details of TC spatial distribution and can deal with different TC scenarios effectively. The effectiveness of our model indicates that the method based on deep learning has strong applicability and development prospects in the TC precipitation nowcasting, providing infinite possibilities and potential for this field. • We develop a novel deep-learning framework for TC precipitation nowcasting. • We propose a multi-source information fusion mechanism between TC and precipitation. • We design an effective dynamic spatio-temporal reasoning graph for TC precipitation. • Our model performs better than other popular deep-learning methods in this task. • Our model improves the high-intensity precipitation nowcasting results remarkably. [ABSTRACT FROM AUTHOR]

• We present a novel Contour-Guided Graph Reasoning Network (CGRNet) for ambiguous biomedical image segmentation, which reduces boundary errors and improves segmentation performance. • We introduce a contour-guided graph reasoning (CGR) module, which updates and propagates cross-domain features using region and contour features as graph nodes to achieve precise segmentation along the boundaries. • We demonstrate our model's effectiveness and impressive performance by comparing our CGRNet with current state-of-the-art methods through extensive experiments on various benchmarks. In this work, we propose to address the existing problem of biomedical image segmentation that often produces results, which fail to capture the exact contours of the target and suffer from ambiguity. Most previous techniques are suboptimal because they often simply concatenate contour information to alleviate this problem, while ignoring the correlation between regions and contours. As a matter of fact, the relationship between cross-domain features is an important clue for ambiguous pixel segmentation in biomedical images. To this end, we contribute a simple yet effective framework called Contour-Guided Graph Reasoning Network (CGRNet) for more accurate segmentation against ambiguity, which is capable of capturing the semantic relations between object regions and contours through graph reasoning. Specifically, we first perform a global graph representation of the low-level and high-level features extracted by the feature extractor, where clusters of pixels with similar features are mapped to each vertex. Further, we explicitly combine contour information as the geometric prior, which can aggregate features of contour pixels to graph vertices and focus on features along the boundaries. Then, the cross-domain features propagate information through the vertices on the graph to efficiently learn and reason about the semantic relations. Finally, the learned refinement graph features are projected back to the original pixel coordinate space for the final pixel-wise segmentation task. Extensive experiments on the three publicly available Kvasir, CVC-612, and COVID19-100 datasets show the effectiveness of our CGRNet with superior performance to existing state-of-the-art methods. Our code is publicly available at: https://github.com/DLWK/CGRNet. [ABSTRACT FROM AUTHOR]

Air pollution is a complex process and is affected by meteorological conditions and other chemical components. Numerous studies have demonstrated that data-driven spatio-temporal prediction models of PM 2.5 concentration are comparable with the model-driven model. However, data-driven models are usually depending on the statistical correlation between PM 2.5 and other factors and have challenges in dealing with causality in complex systems. In this paper, we argue that domain knowledge should be incorporated into data-driven models to enhance prediction accuracy and make the model more physically realistic. We focus on the influence of dynamic wind-field on PM 2.5 concentration distribution and fuse the pollution diffusion distance with the deep learning model based on a wind-field surface. In order to model spatial dependence between monitoring stations, which is dynamic and anisotropic because of the wind-field, we proposed a hybrid deep learning framework, dynamic directed spatio-temporal graph convolution networks (DD-STGCN). It expanded the ability to deal with space-time prediction in the continuous and dynamic wind-field. We used a directed graph time-series to describe the vertex state and topological relationship between vertices and replaced traditional Euclidean distance with wind-field diffusion distance to describe the proximity relationship between vertices. Our experiment results demonstrated that the DD-STGCN model achieved a better prediction ability than LSTM, GC-LSTM, and STGCN models. Compared to the best comparison model, MAPE, MAE, and RMSE were improved by 10.2%, 9.7%, and 9.6% in 12 h on an average, respectively. The performance of our model was further tested during a haze period. In the case that two models both considered the effect of wind, compared with the pure data-driven model, our model performed better in prediction distribution and showed the benefit of spatial interpretability provided by domain knowledge. Image 1 • The influence of dynamic wind-field is considered by embedding the diffusion distance into the deep learning-based model. • The hybrid graph convolution-based model improves the prediction accuracy. • The domain knowledge helps for a more precise spatial relationship and provides spatial interpretations. Summary of the main finding: The consideration of dynamic wind-field improves the PM 2.5 prediction accuracy, and it also helps to describe the spatial relationship more precisely and provides spatial interpretability. [ABSTRACT FROM AUTHOR]