Search

We investigate the problem of statistical analysis of interval-valued time series data – two nonintersecting real-valued functions, representing lower and upper limits, over a period of time. Specifically, we pay attention to the two concepts of phase (or horizontal) variability and amplitude (or vertical) variability, and propose a phase-amplitude separation method. We view interval-valued time series as elements of a function (Hilbert) space and impose a Riemannian structure on it. We separate phase and amplitude variability in observed interval functions using a metric-based alignment solution. The key idea is to map an interval to a point in R 2 , view interval-valued time series as parameterized curves in R 2 , and borrow ideas from elastic shape analysis of planar curves, including PCA, to perform registration, summarization, analysis, and modeling of multiple series. The proposed phase-amplitude separation provides a new way of PCA and modeling for interval-valued time series, and enables shape clustering of interval-valued time series. We apply this framework to three different applications, including finance, meteorology and physiology, proves the effectiveness of proposed methods, and discovers some underlying patterns in the data. Experimental results on simulated data show that our method applies to the point-valued time series. [ABSTRACT FROM AUTHOR]

We develop a multivariate regression model when responses or predictors are on nonlinear manifolds, rather than on Euclidean spaces. The nonlinear constraint makes the problem challenging and needs to be studied carefully. By performing principal component analysis (PCA) on tangent space of manifold, we use principal directions instead in the model. Then, the ordinary regression tools can be utilized. We apply the framework to both shape data (ozone hole contours) and functional data (spectrums of absorbance of meat in Tecator dataset). Specifically, we adopt the square-root velocity function representation and parametrization-invariant metric. Experimental results have shown that we can not only perform powerful regression analysis on the non-Euclidean data but also achieve high prediction accuracy by the constructed model. [ABSTRACT FROM AUTHOR]

We study the problem of classifying actions of human subjects using depth movies generated by Kinect or other depth sensors. Representing human body as dynamical skeletons, we study the evolution of their (skeletons’) shapes as trajectories on Kendall’s shape manifold. The action data is typically corrupted by large variability in execution rates within and across subjects and, thus, causing major problems in statistical analyses. To address that issue, we adopt a recently-developed framework of Su et al. [1] , [2] to this problem domain. Here, the variable execution rates correspond to re-parameterizations of trajectories, and one uses a parameterization-invariant metric for aligning, comparing, averaging, and modeling trajectories. This is based on a combination of transported square-root vector fields (TSRVFs) of trajectories and the standard Euclidean norm, that allows computational efficiency. We develop a comprehensive suite of computational tools for this application domain: smoothing and denoising skeleton trajectories using median filtering, up- and down-sampling actions in time domain, simultaneous temporal-registration of multiple actions, and extracting invertible Euclidean representations of actions. Due to invertibility these Euclidean representations allow both discriminative and generative models for statistical analysis. For instance, they can be used in a SVM-based classification of original actions, as demonstrated here using MSR Action-3D, MSR Daily Activity and 3D Action Pairs datasets. Using only the skeletal information, we achieve state-of-the-art classification results on these datasets. [ABSTRACT FROM PUBLISHER]

• We propose a Transformer-based network architecture for video salient object detection. • We propose a Gated Cross Reference (GCR) module to facilitate collaborative learning between appearance and motion cues. • The proposed Transformer-based method reaches excellent performance on five public benchmarks. Video salient object detection is a fundamental computer vision task aimed at highlighting the most conspicuous objects in a video sequence. There are two key challenges presented in video salient object detection: (1) how to extract effective feature representations from appearance and motion cues, and (2) how to combine both of them into robust saliency representation. To handle these challenges, in this paper, we propose a novel Transformer-based Cross Reference Network (TCRN), which fully exploits long-range context dependencies in both feature representation extraction and cross-modal (i.e., appearance and motion) integration. In contrast to existing CNN-based methods, our approach formulates video salient object detection as a sequence-to-sequence prediction task. In the proposed approach, the deep feature extraction is achieved by a pure vision transformer with multi-resolution token representations. Specifically, we design a Gated Cross Reference (GCR) module to effectively integrate appearance and motion into saliency representation. The GCR first propagates global context information between different modalities, and then perform cross-modal fusion by a gate mechanism. Extensive evaluations on five widely-used benchmarks show that the proposed Transformer-based method performs favorably against the existing state-of-the-art methods [ABSTRACT FROM AUTHOR]

Structural condition identification based on monitoring data is important for automatic civil infrastructure asset management. Nevertheless, the monitoring data are almost always insufficient because the real-time monitoring data of a structure only reflect a limited number of structural conditions, while the number of possible structural conditions is infinite. With insufficient monitoring data, the identification performance may significantly degrade. This study aims to tackle this challenge by proposing a deep transfer learning (TL) approach for structural condition identification. It effectively integrates physics-based and data-driven methods by generating various training data based on the calibrated finite element (FE) model, pretraining a deep learning (DL) network, and transferring its embedded knowledge to the real monitoring/testing domain. Its performance is demonstrated in a challenging case, vibration-based condition identification of steel frame structures with bolted connection damage. First, disparate subsets of test data are used as training data, and the identification accuracy of the whole dataset is evaluated. The results demonstrate that the proposed approach can achieve high identification accuracy with limited types of training data, with the identification accuracy increasing up to 8.57%. Second, numerical simulation data are used as training data, and then different TL strategies and different DL architectures are compared on the performance of structural condition identification. The results show that even though the training data are from a different domain and with different types of labels, intrinsic physics can be learned through the pretraining process, and the TL results can be clearly improved, with the identification accuracy increasing from 81.8% to 89.1%. The comparative studies show that SHMnet with three convolutional layers stands out as the pretraining DL architecture, with 21.8% and 25.5% higher identification accuracy values over the other two networks, VGGNet-16 and ResNet-18. The findings of this study advance the potential application of the proposed approach towards expert-level condition identification based on limited real-world training data. [ABSTRACT FROM AUTHOR]

Visual observations of dynamic phenomena, such as human actions, are often represented as sequences of smoothly-varying features. In cases where the feature spaces can be structured as Riemannian manifolds, the corresponding representations become trajectories on manifolds. Analysis of these trajectories is challenging due to non-linearity of underlying spaces and high-dimensionality of trajectories. In vision problems, given the nature of physical systems involved, these phenomena are better characterized on a low-dimensional manifold compared to the space of Riemannian trajectories. For instance, if one does not impose physical constraints of the human body, in data involving human action analysis, the resulting representation space will have highly redundant features. Learning an effective, low-dimensional embedding for action representations will have a huge impact in the areas of search and retrieval, visualization, learning, and recognition. Traditional manifold learning addresses this problem for static points in the euclidean space, but its extension to Riemannian trajectories is non-trivial and remains unexplored. The difficulty lies in inherent non-linearity of the domain and temporal variability of actions that can distort any traditional metric between trajectories. To overcome these issues, we use the framework based on transported square-root velocity fields (TSRVF); this framework has several desirable properties, including a rate-invariant metric and vector space representations. We propose to learn an embedding such that each action trajectory is mapped to a single point in a low-dimensional euclidean space, and the trajectories that differ only in temporal rates map to the same point. We utilize the TSRVF representation, and accompanying statistical summaries of Riemannian trajectories, to extend existing coding methods such as PCA, KSVD and Label Consistent KSVD to Riemannian trajectories or more generally to Riemannian functions. We show that such coding efficiently captures trajectories in applications such as action recognition, stroke rehabilitation, visual speech recognition, clustering and diverse sequence sampling. Using this framework, we obtain state-of-the-art recognition results, while reducing the dimensionality/complexity by a factor of $100-250\times$ . Since these mappings and codes are invertible, they can also be used to interactively visualize Riemannian trajectories and synthesize actions. [ABSTRACT FROM AUTHOR]

Time series prediction has been widely applied to the finance industry in applications such as stock market price and commodity price forecasting. Machine learning methods have been widely used in financial time series prediction in recent years. How to label financial time series data to determine the prediction accuracy of machine learning models and subsequently determine final investment returns is a hot topic. Existing labeling methods of financial time series mainly label data by comparing the current data with those of a short time period in the future. However, financial time series data are typically non-linear with obvious short-term randomness. Therefore, these labeling methods have not captured the continuous trend features of financial time series data, leading to a difference between their labeling results and real market trends. In this paper, a new labeling method called "continuous trend labeling" is proposed to address the above problem. In the feature preprocessing stage, this paper proposed a new method that can avoid the problem of look-ahead bias in traditional data standardization or normalization processes. Then, a detailed logical explanation was given, the definition of continuous trend labeling was proposed and also an automatic labeling algorithm was given to extract the continuous trend features of financial time series data. Experiments on the Shanghai Composite Index and Shenzhen Component Index and some stocks of China showed that our labeling method is a much better state-of-the-art labeling method in terms of classification accuracy and some other classification evaluation metrics. The results of the paper also proved that deep learning models such as LSTM and GRU are more suitable for dealing with the prediction of financial time series data. [ABSTRACT FROM AUTHOR]

Cloud Computing (CC) makes it possible for a common user to get an access to large pools of data and computational resources through a variety of interfaces. Among the so many important problems in CC, load balancing technique has been paid more and more attention for its important role. Good load balance algorithms can make whole system run more efficient. A new load balancing method combined with the advantage of online and offline load balancing algorithms are proposed in this paper. Two-choice algorithm and its improvement are used in the online step. Bacteria Foraging Optimization (BFO) and its improvement motivated by Lamarck Evolutionary Theory are introduced in our offline step. Online load balancing uses imperfect information, aiming at finishing tasks as fast as possible; while the offline makes full use of all information to make a supplement. Experiments on the heterogeneous tasks and serving points for computation intensive loads have been used here and the good performance shows the efficiency of our proposed method. [ABSTRACT FROM AUTHOR]

Activity interpretation in videos results not only in recognition or labeling of dominant activities, but also in semantic descriptions of scenes. Towards this broader goal, we present a combinatorial approach that assumes availability of algorithms for detecting and labeling objects and basic actions in videos, albeit with some errors. Given these uncertain labels and detected objects, we link them into interpretable structures using the domain knowledge, under the framework of Grenander's general pattern theory. Here a semantic description is built using basic units, termed generators, that represent either objects or actions. These generators have multiple out-bonds, each associated with different types of domain semantics, spatial constraints, and image evidence. The generators combine, according to a set of pre-defined combination rules that capture domain semantics, to form larger configurations that represent video interpretations. This framework derives its representational power from flexibility in size and structure of configurations. We impose a probability distribution on the configuration space, with inferences generated using a Markov chain Monte Carlo-based simulated annealing process. The primary advantage of the approach is that it handles known challenges-appearance variabilities, errors in object labels, object clutter, simultaneous events, etc-without the need for exponentially-large (labeled) training data. Experimental results demonstrate its ability to successfully provide interpretations under clutter and the simultaneity of events. They show: (1) a performance increase of more than 30 % over other state-of-the-art approaches using more than 5000 video units from the Breakfast Actions dataset, and (2) an overall recall and precision improvement of more than 50 and 100 %, respectively, on the YouCook data set. [ABSTRACT FROM AUTHOR]

We develop a combinatorial approach to represent and infer semantic interpretations of video contents using tools from Grenander’s pattern theory. Semantic structures for video interpretation are formed using generators and bonds, the fundamental units of representation in pattern theory. Generators represent features and ontological items, such as actions and objects, whereas bonds are threads used to connect generators while respecting appropriate constraints. The resulting configurations of partially-connected generators are termed scene interpretations. Our goal is to parse a given video data set into high-probability configurations. The probabilistic models are imposed using energies that have contributions from both data (classification scores) and prior information (ontological constraints, co-occurrence frequencies, etc). The search for optimal configurations is based on an MCMC, simulated-annealing algorithm that uses simple moves to propose configuration changes and to accept/reject them according to the posterior energy. In contrast to current graphical methods, this framework does not preselect a neighborhood structure but tries to infer it from the data. The proposed framework is able to obtain 20% higher classification rates, compared to a purely machine learning-based baseline, despite artificial insertion of low-level processing errors. In an uncontrolled scenario, video interpretation performance rates are found to be double that of the baseline. [ABSTRACT FROM AUTHOR]

• We propose a novel multi-view DCNN-based approach which can take advantage of not only multi-view images but also the relationships between them. • The proposed networks learn the integrated features of multi-view fundus images for DR detection. • We introduce the attention mechanisms for mining the relationship between different views. • In order to boost the ability to capture the tiny lesion features in the retinal images, we combine the idea of attention mechanisms with the channel dimension. Diabetic retinopathy (DR) detection has attracted much attention recently, and the deep learning algorithms have gained traction in this area. At present, DR screening by deep learning algorithms is often based on single-view fundus images, which usually leads to an unsatisfactory accuracy of DR grading due to the incomplete lesion features. In this paper, we proposed a novel diabetic retinopathy detection convolutional network for automatic DR detection by integrating multi-view fundus images. Compared to existing single-view DCNN-based DR detection methods, the proposed method has the following advantages. First, our method fully utilizes the lesion features from the retina with a field-of-view around 120 ∘ − 150 ∘. Second, by introducing the attention mechanisms, more attention will be paid on the influential view and the performance can be improved. Besides, we also assign large weights to important channels in the network for effective feature extraction. Experiments are conducted on our collected multi-view DR dataset contained 15,468 images, in which each eye sample provides four-view images. The experimental results indicate that using multi-view images is suitable for automatic DR detection and our proposed method is superior to other benchmarking methods. [ABSTRACT FROM AUTHOR]