Search

Video captioning is a joint task of computer vision and natural language processing, which aims to describe the video content using several natural language sentences. Nowadays, most methods cast this task as a mapping problem, which learns a mapping from visual features to natural language and generates captions directly from videos. However, the underlying challenge of video captioning, i.e., sequence to sequence mapping across the different domains, is still not well handled. To address these problems, we introduce the polishing mechanism in an attempt to mimic human polishing process and propose a generate-and-polish framework for video captioning. In this paper, we propose a two-step transformer based polishing network (TSTPN) consisting of two sub-modules: the generation-module is to generate the caption candidate and the polishing-module is to gradually refine the generated candidate. Specifically, the candidate provides a global information of the visual contents in a semantically-meaningful order, where it is firstly considered as a semantic intersnubber to bridge the semantic gap between the text and video, with the cross-modal attention mechanism for better cross-modal modeling; and it secondly provides a global planning ability to maintain the semantic consistency and fluency of the whole sentence for better sequence mapping. In experiments, we present adequate evaluations to show that the proposed TSTPN achieves the comparable and even better performance than the state-of-the-art methods on the benchmark datasets. [ABSTRACT FROM AUTHOR]

Labanotation is a symbolic recording system for human movements, and also a powerful tool for protecting and spreading folk dances and other performing arts. State-of-the-art automatic Labanotation uses end-to-end methods with sequence-based skeleton representation, which cannot capture the relationship between joints and bones in the skeleton for accurate descriptions of continuous lower limb movements such as dance steps. In this paper, we propose a novel double-stream fusion method of directed graph neural networks (DGNN), combined with connectionist temporal classification (CTC), namely DFGNN-CTC, for sequential fine-grained motion recognition, such as the Labanotation generation of unsegmented dance movement. First, we extract double-stream directed graph feature, employing an orientation-normalized directed acyclic graph (ON-DAG) and an orientation-normalized temporal directed acyclic graph (ON-TDAG), to jointly model spatiotemporal properties of movement recorded in motion capture data. Then, we design a CTC-based fusion-pooling module to fuse the spatial and temporal streams encoded by two DGNNs. It concatenates and fuses the two streams to generate discriminative descriptions of each time step, and concentrates them to make per-time-step predictions of Laban gesture type, from which the CTC searches the optimal Laban symbol sequence, corresponding to elemental motions composing the movement. In this way, the new method enables much finer discrimination for similar Laban gestures with subtle differences in spatial and temporal properties through joint contextual spatiotemporal modeling so that it achieves much superior performance in continuous Labanotation generation to existing methods, which only have single-stream analysis either spatially or temporally. The experiments on two Labanotation-labelled motion capture datasets demonstrate the effectiveness of the components in the proposed method and its superiority comparing with the state-of-the-art methods, especially for lower limb movements. [ABSTRACT FROM AUTHOR]

Labanotation is a widely-used notation system for dance recording. Numerous methods for automatic Labanotation generation from motion capture data have been proposed to save time and human labor. However, dance performance variations, data noises, and difficulties in recognizing long continuous motion sequences limit the performance of existed methods. In this paper, we propose a CRNN-based attention-seq2seq model with fusion features for a robust and effective Labanotation generation.First, we fuse bone feature and Lie group feature to extract not only the information of bones between adjacent joints but also the relative geometry relationships between connected bones. Then, in the proposed seq2seq model, we employ the Convolutional Recurrent Neural Networks (CRNN) to learn the spatial–temporal representation of motion capture data and the attention mechanism to learn good alignments between input motion feature sequences and output symbol sequences. Extensive experiments on real-world datasets show that the proposed method obtains considerable recognition accuracy (90.65% on the LabanSeq16 dataset and 93.29% on the LabanSeq48 dataset), which outperforms state-of-the-art approaches on the task of automatic Labanotation generation. [ABSTRACT FROM AUTHOR]

Human activity localization aims at recognizing contents and detecting locations of activities in video sequences. With an increasing number of untrimmed video data, traditional activity localization methods always suffer from two major limitations. First, detailed annotations are needed in most existing methods, i.e., bounding-box annotations in every frame, which are both expensive and time consuming. Second, the search space is too large for 3D activity localization, which requires generating a large number of proposals. In this paper, we propose a unified deep Q-network with weak reward and weak loss (DWRLQN) to address the two problems. Certain weak knowledge and weak constraints involving the temporal dynamics of human activity are incorporated into a deep reinforcement learning framework under sparse spatial supervision, where we assume that only a portion of frames are annotated in each video sequence. Experiments on UCF-Sports, UCF-101 and sub-JHMDB demonstrate that our proposed model achieves promising performance by only utilizing a very small number of proposals. More importantly, our DWRLQN trained with partial annotations and weak information even outperforms fully supervised methods. [ABSTRACT FROM AUTHOR]

Abstract Locating human action in spatio-temporal domain among untrimmed videos is an important but challenging task. Recent works have shown that incorporating contextual information leads to a significant improvement in action recognition, but there is still no existing work taking full advantage of context for action localization. While the popular target-centered methods have achieved promising results, they fail to exploit contexts and capture temporal dynamics in actions. In this paper, we propose a principled dynamic model, called spatio-temporal context model (STCM), to simultaneously locate and recognize actions. The STCM integrates various kinds of contexts, including the temporal context that consists of the sequences before and after action as well as the spatial context in the surrounding of target. Meanwhile, a novel dynamic programming approach is introduced to accumulate evidences collected at a small set of candidates in order to detect the spatio-temporal location of action effectively and efficiently. We report encouraging results on the UCF-Sports and UCF-101. It demonstrates that the contextual information is not only helpful for action recognition, but also contributes to action localization. [ABSTRACT FROM AUTHOR]

Labanotation is one of the most widely used notation systems and plays a powerful role in recording and archiving traditional folk dance. We propose an end-to-end method for generating Labanotation from motion capture data by identifying the movement of each part of the human body and by assigning corresponding Labanotation symbols. Our method is mainly highlighted in the following aspects: first, we design simple yet highly discriminative skeleton features that can accurately represent human movements; and second, for the recognition of upper limb movements, we adopt fast and efficient extreme-learning neural networks, and for the recognition of lower limb movements, we employ powerful long short-term memory networks. It is worth mentioning that this is the first time that neural networks have been applied to the field of Labanotation generation. Experimental results show that our approach achieves much better recognition accuracy than previous work. © 2019 SPIE and IS&T [DOI: 10.1117/1.JEI.28.2.023026] [ABSTRACT FROM AUTHOR]

Zero-shot learning (ZSL) has recently attracted increasing attention in visual tasks like action recognition. We propose a spatiotemporal visual-semantic embedding network (STVSEM) for zero-shot action recognition. First, given the fact that two-stream architecture based action recognition algorithms have achieved excellent results in recent years, the module is assembled to our designed network by simultaneously using the spatial features (e.g., RGB appearance) and optical flow in time domain as visual features to significantly improve the visual expression capability. Then, in order to slightly alleviate the problem of semantic loss that typically occurs in the case of using embedding-based ZSL methods, an autoencoder is introduced to get a better semantic representation and complement semantic relationship information for unseen classes by seen classes. Last but not least, a joint embedding mechanism that explores and exploits the relationships of the visual data and semantic information in an intermediate space is employed to ameliorate the gap between vision and semantics. The experimental results on Charades and UCF101 datasets indicate that the proposed method outperforms the state-of-the-art methods in accuracy, which further demonstrates the effectiveness of our method. [ABSTRACT FROM AUTHOR]

Video denoising is a low-level task that uses the information of neighbor frames to denoise polluted frames. Using the information provided by neighboring frames, noise can be effectively removed and the denoising effect is improved. Through continuous development, single-stage and multi-stage denoising methods have been proposed. However, these existing denoising methods simply use the information provided by neighbor frames to denoise the polluted frames, paying insufficient attention to the polluted frames and making insufficient use of the information of the polluted frames. We propose a two-stage video denoising network called multiple forward-backward strategies for two-stage video denoising, which improves the denoising performance and retains more detailed information by enhancing the utilization of information of polluted frames. Extensive experiments have been conducted, and the experimental results show that the proposed method outperforms the state-of-the-art denoising methods in both qualitative and quantitative comparisons. [ABSTRACT FROM AUTHOR]

Labanotation is a widely-used notation system for recording human dance movements. Automatically generating Labanotation scores from motion capture data can save significant manual effort and help the preservation of old folk dances in protecting intangible cultural heritages. Existing Labanotation generation methods have limited ability to capture the flexible limb movements as well as the rich periodic, symmetric, or repeated dance steps. In this paper, we present a novel LabanFormer model including a Multi-Scale Graph Attention network (MS-GAT) and a transformer model with Gated Recurrent Positional Encoding (GRPE) to achieve more effective Labanotation generation. First, the proposed MS-GAT can capture flexible limb movements by learning feature correlations between every two joints and aggregating features of neighboring joints over multiple scales. Second, we propose a new GRPE-based transformer to learn global temporal dependencies in the output feature sequences of MS-GAT. The novel GRPE module can encode position information with learnable parameters while handling various sequence lengths. As such, the periodic, symmetric, or repeated steps in dances can be accurately captured. Finally, the corresponding Laban symbols are generated by the decoder of the GRPE-based transformer. Extensive experiments on two real-world datasets show that the proposed LabanFormer model obtains remarkable performance compared with state-of-the-art approaches on the automatic Labanotation generation task. [ABSTRACT FROM AUTHOR]

Recent works have shown that mid-level feature is superior to low-level feature, which can not only improve discriminative power, but also enhance descriptive capability. In this paper, the classical STIP, spatial star-graph and temporal star-graph are first extracted to represent human action from multi-perspectives. Then a principled feature learning algorithm is proposed to embed these multi-cues into a unified space and enhance all low-level features using diffusion map. Unlike treating spatio-temporal patch as mid-level primitive, we use a graph to model different types of primitives, then apply graph partitioning to co-cluster them into visual-word clusters called mid-level distinctive feature, which can bridge the semantic gap across low-level features. Experimental results show that our approach can successfully classify human activities with higher accuracies both on single-person actions (KTH and UCF) and complex interactional activities (UT-Interaction and HMDB51). [ABSTRACT FROM AUTHOR]

Linking together two directional entropy disequilibriums, NOR functionality can be found. The entropy NOR gate presented is constructed of discrete observations and so is very small, emerging at the earliest stages of complexity. The gate is based on the axiom that an observer increases in entropy as it receives information from what it is observing. [ABSTRACT FROM AUTHOR]

Abnormal events detection plays an important role in the video surveillance, which is a challenging subject in the intelligent detection. In this paper, based on a novel motion feature descriptor, that is, the histogram of maximal optical flow projection (HMOFP), we propose an algorithm to detect abnormal events in crowded scenes. Following the extraction of the HMOFP of the training frames, the one-class support vector machine (SVM) classification method is utilized to detect the abnormality of the testing frames. Compared with other methods based on the optical flow, experiments on several benchmark datasets show that our algorithm is effective with satisfying results. [ABSTRACT FROM AUTHOR]

In this study, we present a new image completion method based on image entropy reduction. We complete the missing region with semantically matching images, which maximizes the reduction in the combined entropy of all regions in the image. We use labeled regions (high confidence regions) to complete the uncertain regions. By contrast, existing image completion methods focus on simple filling and ignore creative and semantic matching completion. Entropy reduction can yield higher accuracy of semantic image matching than existing image completion methods. We use Poisson blending and blending optimization (color handling) to complete the missing region with higher-quality results. The superiority of our method to existing image completion algorithms is validated. Experiments using an image database show that our method significantly improves the completion results. [ABSTRACT FROM AUTHOR]

In this paper, we present a new object recognition model with few instances based on semantic distance. Learning objects with many instances have been studied in computer vision for many years. However, in many cases, not enough positive instances occur, especially for some special categories. We must take full advantage of all instances, including those that do not belong to the category. The main insight is that, given a few positive instances from one category, we can define some other candidate instances as positive instances based on semantic distance to learn this model. Our model responds more strongly to instances with closer semantic distance to positive instances than to instances with farther semantic distance to positive instances. We use a regularized kernel machine algorithm to train the images from the database. The superiority of our method to existing object recognition methods is demonstrated. Experiments using an image database show that our method not only reduces the number of learning instances but also keeps the accurate rate of recognition. [ABSTRACT FROM AUTHOR]

We study the task of object recognition with no or few instances in its category. There are thousands of categories and only a few attributes. In other words, some attributes can be shared with different categories. In some case, it is difficult for some special categories to search enough positive instances. Even if it is easy for some category to search enough positive instances, it still waste time and resources. In this paper, we present an innovative image recognition method with no or few learning instances. We use attributes shared by other categories to learn the new category. It can make full use of the attributes and solve the problem that some categories do not have enough positive learning instances. Other objects or scenes in image also make contributions to improve recognition. At last, we set up a new database to do some experiments. The experimental results show our method's superiority. Although our method learns new category with no or few positive instances, it still keeps the high accuracy compared to traditional methods. [ABSTRACT FROM AUTHOR]

In this paper, we present a new model applied to watershed segmentation optimization. Watershed segmentation is widely used in the domain of computer vision. However, in some cases, there are some unobvious gradient changing between different regions or obvious gradient changing in the same region. So the images are divided into a number of regions either smaller (under-segmentation) or larger (over-segmentation) than the expected numbers of regions. We use a new filter (OLS operator) to pre-process the image. The new filter can smooth the image but keep the edge between different regions sharper based on image decompositions. Compared to traditional watershed segmentation, our method can segment images more accurately. Experiments using an image database show the superiority of our method. Even if we apply our method to complex images with more regions, it is still effective. Furthermore, our model can be applied to other domains of computer vision (e.g., image classification, image recognition, image matting). [ABSTRACT FROM AUTHOR]

In this paper, we propose a new image completion method based on high-accuracy semantic matching. Existing image completion approaches mainly focus on simple filling, ignoring creativity and accuracy in semantic matching. Our method can complete missing regions with more creative and semantically matching images as well as create a more seamless and consistent completed image. We use global and local features to search matching images for the target image. We then complete the missing region using Poisson blending and blending optimization. Results of experiments on challenging image databases show that the images are greatly improved. Thus, these results demonstrate the superiority of the proposed algorithm over existing image completion approaches. [ABSTRACT FROM AUTHOR]

As standard probabilistic latent semantic analysis (pLSA) is oriented to discrete quantity only, pLSA with Gaussian mixtures (GM-pLSA) succeeding in transferring it to continuous feature space is proposed, which uses Gaussian mixture model to describe the feature distribution under each latent aspect. However, inheriting from pLSA, GM-pLSA still overlooks the intrinsic interdependence between terms, which indeed is an important clue for performance improvement. In this paper, we present a graph regularized GM-pLSA (GRGM-pLSA) model as an extension of GM-pLSA to embed this term correlation information into the process of model learning. Specifically, grounded on the manifold regularization principle, a graph regularizer is introduced to characterize the correlation between terms; by imposing it on the objective function of GM-pLSA, model parameters of GRGM-pLSA are derived via corresponding expectation maximization algorithm. Furthermore, two applications to video content analysis are devised. One is video categorization where GRGM-pLSA serves for feature mapping with two kinds of sub-shot correlations, respectively, incorporated, while the other provides a new perspective on video concept detection, which transforms the detection task to a GRGM-pLSA-based visual-to-textual feature conversion problem. Extensive experiments and comparison with GM-pLSA and several state-of-the-art approaches in both applications demonstrate the effectiveness of GRGM-pLSA. [ABSTRACT FROM AUTHOR]

Low-resolution face recognition (LR FR) aims to recognize faces from small size or poor quality images with varying pose, illumination, expression, etc. It has received much attention with increasing demands for long distance surveillance applications, and extensive efforts have been made on LR FR research in recent years. However, many issues in LR FR are still unsolved, such as super-resolution (SR) for face recognition, resolution-robust features, unified feature spaces, and face detection at a distance, although many methods have been developed for that. This paper provides a comprehensive survey on these methods and discusses many related issues. First, it gives an overview on LR FR, including concept description, system architecture, and method categorization. Second, many representative methods are broadly reviewed and discussed. They are classified into two different categories, super-resolution for LR FR and resolution-robust feature representation for LR FR. Their strategies and advantages/disadvantages are elaborated. Some relevant issues such as databases and evaluations for LR FR are also presented. By generalizing their performances and limitations, promising trends and crucial issues for future research are finally discussed. [ABSTRACT FROM AUTHOR]

In this paper, we focus on the improvements of appearance representation for multihuman tracking. Many previous methods extracted low-level appearance features, such as color histogram and texture, even combined with spatial information for each frame. These methods ignore the temporal distribution of features. The features of each frame may not be stable due to illumination, human pose variation, and image noise. In order to improve it, we propose a novel appearance representation called the spatial–temporal appearance model based on the statistical distribution of Gaussian mixture model (GMM). It represents the appearance of a tracklet as a whole with dynamic spatial and temporal information. The spatial information is the dynamic subregions. The temporal information is the dynamic duration time of each subregion. Each subregion is modeled as the weighted Gaussian distribution of GMM. The online expectation-maximization (online EM) algorithm is used to estimate the parameters of GMM. Then, we propose a tracklet association method using Bayesian prediction and Jensen–Shannon divergence. The Bayesian prediction is used to predict the locations of targets. The Jensen–Shannon divergence is used to compute the distance of spatial–temporal appearance distribution between two tracklets. Finally, we test our approach on four challenging datasets (TRECVID, CAVIAR, ETH, and EPFL Terrace) and achieve good results. [ABSTRACT FROM PUBLISHER]

Multiperson activity recognition in videos is a challenging task, due to the complexity of interactions among multiple persons. In this paper, a new statistical model, named coupled observation decomposed hidden Markov model (CODHMM), is presented to model multiperson activities in videos. A human activity that involves multiple persons is analyzed in two levels: the individual level that describes each individual's motion details and the interaction level that expresses the shared information among multiple persons. The two levels are modeled by two hidden Markov chains that are interdependent and interact with each other. The observation in each chain at each time slice is decomposed into subobservations according to the number of features and the number of persons. For each activity to be recognized, a CODHMM is built and model parameters are learnt by a generalized expectation maximization (EM) algorithm. Given an input video that contains an unknown activity, maximum likelihood algorithms are developed to classify it into one of the learnt activity categories. Experimental results show that the CODHMM can successfully classify human activities involving multiple persons with high accuracy and low computations. [ABSTRACT FROM AUTHOR]

In this paper, a novel video matting method based on opacity propagation is proposed. It adopts a two-step framework by first segmenting each frame into bilayer and generating trimaps, and then applying matting algorithm to extract the final alpha mattes. In the bilayer segmentation step, an opacity propagation algorithm is used to predict the foreground object in the following frame. It propagates the opacity values of the former frame to the current, and generates an Opacity Map (OM), which is in most cases much more accurate than the previously used Probability Map (PM). OM is used to extract the foreground object by combining it with the graph cut algorithm. And then an accurate trimap is generated by a trimap refinement method based on a Local Gaussian Mixture Model (LGMM). At last, the opacity propagation is again applied to the unknown regions of the trimap to generate the accurate alpha mattes in a temporally coherent way. The experiment results demonstrate the effectiveness of our method. [ABSTRACT FROM AUTHOR]

Abstract: Given a segmentation result (an alpha matte or a binary mask) of the former frame, foreground prediction is a process of estimating the probability that each pixel in the current frame belongs to the foreground. It plays a very important role in bilayer segmentation of videos, especially videos with non-static backgrounds. In this paper, a new foreground prediction algorithm which is called opacity propagation is proposed. It can propagate the opacity values of the former frame to the current frame by minimizing a cost function that is constructed by assuming the spatiotemporally local color smoothness of the video. Optical flow and probability density estimation based on a local color model are employed to find the corresponding pixels of two adjacent frames. An OPSIC (opacity propagation with sudden illumination changes) algorithm is also proposed which is an improvement of our proposed opacity propagation algorithm because it adds a simple color transformation model. As far as we know, this is the first algorithm that can predict the foreground accurately when the illumination changes suddenly. The opacity map (OM) generated by the opacity propagation algorithm is usually more accurate than the previously used probability map (PM). The experiments demonstrate the effectiveness of our algorithm. [Copyright &y& Elsevier]

Abstract: Herein an effective method for fabrication of three dimensional networks of starch is presented. In this method, the native starch was dissolved in hot water to form solution, resulting in the formation of starch gel after cooled to room temperature. Then the water in the starch gel was exchanged by ethanol, followed by drying with supercritical (SC) CO2 extraction, the nano-networks of starch were prepared. The obtained starch aerogel was composed of starch fibers and presented porous structure. Changing the initial concentration of native starch and controlling the drying conditions could control the microstructure of the starch networks. The resulting starch networks were utilized as a directing template to prepare the hierarchically ordered TiO2 networks. In this process, TiO2 precursor dissolved in SC CO2 diffused onto the surface of starch gel, and reacted with the hydroxyl groups and adsorbed water molecules on the starch surface, to form an organic/inorganic hybrid material. After calcining the composites at 500°C, the networks composed of anatase TiO2 nanocrystals were produced, which accurately replicated the fine structures of the starch gel and possessed higher BET surface area and total pore volume compared with those of other TiO2 materials templated by some biomaterials or polymer gels. [Copyright &y& Elsevier]

Abstract: In this work, a green route to prepare TiO2 nanotube networks using swim bladder (SB) membrane as a bio-template was reported. In this method, the SB membrane was first dried with CO2 extraction, which resulted in the fibrous structures of SB membrane. With the CO2-dried SB membrane as bio-template, TiO2 were obtained via the surface sol–gel reaction of titanium isopropoxide on the fibers of SB membrane in supercritical CO2, followed by the removal of SB membrane calcined at 600°C. The resulting TiO2 was characterized with scanning electron microscopy, transmission electron microscopy and X-ray diffraction. It was indicated that ordered anatase nanotube networks were produced. Compared to the conventional sol–gel method, the protocol adopted in this work could produce TiO2 nanotubes with regular and ordered structures. [Copyright &y& Elsevier]

Abstract: Mesoporous TiO2 films were successfully prepared via calcining ionic liquid solution containing titanium tetrabutyloxide and cellulose. The as-prepared TiO2 films were characterized by means of different techniques including X-ray diffraction, transmission electron microscopy, scanning electron microscopy and nitrogen sorption analysis. It was indicated that the resulting TiO2 films were composed of anatase nanocrystals with size of about 14nm, and the pore size of the films could be adjusted by the concentration of cellulose in IL. The possible formation mechanism of TiO2 films was also discussed. The prepared anatase films showed a high catalytic activity in photocatalytic reduction of Ag(I) to Ag and Au(III) to Au(0) under UV irradiation. [Copyright &y& Elsevier]

Abstract: This paper describes a rose variety recognition program and proposes Modified Fourier Descriptor. Based on the modified descriptor, a new measurement, Angle Measurement, is proposed for image shape analysis. The rationality of this measurement is analyzed and its mathematical definition is given. We use it for rose flower shape analysis and the experiment results show that this measurement is quite efficient for image shape analysis. [Copyright &y& Elsevier]

• To remove the low variations and noise of objects in the background, we extract the motion descriptor of the foreground by integrating background subtraction with binarization of surveillance videos. • In the training stage, to obtain a low-rank dictionary based on the similarity of normal training samples and a compact cluster of reconstruction coefficient vectors surrounding a center in the meantime, we propose a new joint optimization of the nuclear-norm and l 2, 1 -norm. • In the detection stage, to obtain a large gap between the reconstruction errors of abnormal testing samples and those of normal testing samples, we force the reconstruction coefficient vectors of abnormal frames to distribute so that they resemble those of normal ones by solving an l 2, 1 -norm optimization problem. In this paper, a novel approach to abnormal event detection in crowded scenes is presented based on a new low-rank and compact coefficient dictionary learning (LRCCDL) algorithm. First, based on the background subtraction and binarization of surveillance videos, we construct a feature space by extracting the histogram of maximal optical flow projection (HMOFP) feature of the foreground from a normal training frame set. Second, in the training stage, a new joint optimization of the nuclear-norm and l 2, 1 -norm is applied to obtain a compact coefficient low-rank dictionary. Third, in the detection stage, l 2, 1 -norm optimization is utilized to obtain the reconstruction coefficient vectors of the testing samples. Note that the l 2, 1 -norm forces the reconstruction vectors of all the testing samples to compactly surround the same center in the training stage, such that the reconstruction errors of abnormal testing samples are different from those of normal ones. Finally, a reconstruction cost (RC) is introduced to detect abnormal frames. Experimental results on both global and local abnormal event detection show the effectiveness of our algorithm. Based on comparisons with state-of-the-art methods employing various criteria, the proposed algorithm achieves comparable detection results. [ABSTRACT FROM AUTHOR]

The video captioning task is generating description sentences by learning semantic information. It has a wide range of applications in areas such as video retrieval, automatic generation of subtitles and blind assistance. Visual semantic information plays a decisive role in video captioning. However, traditional methods are relatively rough for video feature modeling, failing to harness local and global features to understand temporal and spatial relationships. In this paper, we propose a video captioning model based on the Transformer and GCN network called “Relation-Enhanced Spatial–Temporal Hierarchical Transformer” (RESTHT). To address the above issues, we present a spatial–temporal hierarchical network framework to jointly model local and global features in terms of both time and space. For temporal modeling, our model learns the direct interactions between diverse video features and sentence features in the temporal sequence via the pre-trained GPT2, and the global feature construction encourages it to capture essential and relevant information. For spatial modeling, we use self-attention and GCN networks to learn the spatial relationship from appearance and motion perspectives jointly. Through spatial–temporal modeling, our method can comprehend the global time–space relationships of complex events in videos and catch the interaction between different objects to generate more accurate descriptions applicable to universal video captioning tasks. We conducted experiments on two widely used datasets, and especially in the MSVD dataset, our model improves the score of CIDEr by 6.1 compared to the baseline and excels present methods by 13. The results verify that our model can fully model the temporal and spatial relationship and outperforms other related models. [ABSTRACT FROM AUTHOR]

Within-image co-saliency detection is to detect/highlight the common saliency (similar-appearance salient objects) in a single image. Ideally, it can be solved by detecting each individual salient object first and then comparing them, which is possible for some images with simple representations. However, in practice, this way is not accurate and robust for some images with complex representations. In this paper, we propose an easy-to-hard learning strategy to solve this problem. By directly localizing and comparing salient objects in simple images, superpixel confidences as co-salient objects are inferred by an easy learning method, which provide promising but also noisy supervisions for complex images. Therefore, within-image co-saliency detection in complex images can be modeled as a hard learning problem with noisy labels. A multi-scale Multiple Instance Learning (MIL) model together with a new sampling method is proposed to solve this hard learning problem with noisy labels. Experimental results show that the proposed method achieves the best performance on a public benchmark dataset and two synthetic datasets. [ABSTRACT FROM AUTHOR]

MgAl2O4 spinel powders were first applied for aluminum dihydrogen phosphate hydrosol curing reaction. The solidification of spinel powders to phosphate hydrosol was studied in middle and low temperature condition. Besides, composites were prepared with quartz textile as reinforcing part, aluminum hydrogen phosphate hydrosol as matrix and spinel MgAl2O4 as firming agent. Curing temperature of phosphate hydrosol and the additive content of spinel MgAl2O4 powders were measured by thermogravimetric analysis and differential scanning calorimeter. At the same time, the crystalline phase of curing products was identified by X-ray powders diffraction. Microstructures of the composites were observed by scanning electron microscope. Moreover, the mechanical and resistant moisture absorption properties of phosphate composites were investigated. The results showed that aluminum dihydrogen phosphate resin had occurred dehydration reaction and formed into cross-linkage net in the temperature near to 170 °C, while the resin was added spinel MgAl2O4 powders. And it was contributed to reduce original pores and flaws in the matrix with mild reaction process. Owing to the promoting effect of spinel, the mainly reaction products were aluminum phosphate with the style of quartz and magnesium phosphate with monoclinic structure. Meanwhile, the reaction that aluminum dihydrogen phosphate changed into aluminum polyphosphate was prevented, which was benefit to enhance the resistant moisture absorption property of the composite. When the content of spinel MgAl2O4 powders arrived at 18wt%, the composite obtained the highest properties, which the flexual strength at room temperature and at 800 °C were about 165 MPa and 75 MPa. And dielectric properties of the composite were remained stably during long time storage, which dielectric constant was about 3.5 to 3.6 and loss tangent was less than 0.01 at room temperature respectively. [ABSTRACT FROM AUTHOR]

As a major research field in computer vision, automatic generation of Labanotation has been attracting the attention of many researchers. These researchers use various methods to engage in dance education and protection, but the existing methods do not consider the spatial-temporal dependence modeling of dance movements. Therefore, they do not represent complex dance movements optimally. We propose an automatic generation network of Labanotation based on the long and short spatial-temporal relations. It includes local spatial-temporal feature extraction network, global spatial-temporal feature extraction network, and local and global feature fusion network. It can align the input and output sequences while modeling the skeletal spatial-temporal relation. The local spatial-temporal features among short interval skeleton sequence relationship are obtained through the multi-scale convolution of time and space. The global spatial-temporal features are learned through the transformer network to obtain the relationship among skeleton sequences with long interval. For the output of the two networks, we use the pyramid squeeze attention network to exchange information in long and short spatial-temporal information, achieving complementarity to improve the accuracy of action recognition. Experimental results show that the proposed method outperforms the state-of-the-art methods on laban16 and laban48, which are common datasets for the research of Labanotation. [ABSTRACT FROM AUTHOR]

Image matting is the extraction of the foreground from an image through the use of provided information. It has been an important technique in the image and video editing field. Current image matting methods estimate the foreground and background, based on information provided regarding the nearby pixels. Color sampling has been an effective means for matting directly, and quite a few methods have achieved high quality matting results based on color sampling. However, there are some drawbacks; for example it is easy to overlook important candidate pixels for matting, and even if the candidate pixels are effectively selected, similar foregrounds and backgrounds will reduce the accuracy of the matting. In this paper’s work, a Weighted-Least Squares (WLS) filter was utilized to sharpen the boundaries between the foregrounds and backgrounds, which facilitated the matting process; and an innovative sampling criterion based on random searching for the matting was then presented. This innovative method could effectively prevent valid samples being overlooked, and could manage the relationships of the nearby and distant pixels. In this process, a new cost function was utilized to evaluate the candidate samples. Experiments utilizing an image database demonstrated that this method significantly improved the matting results. [ABSTRACT FROM AUTHOR]

Image compositing techniques are primarily utilized to achieve realistic composite results. Some existing image compositing methods, such as gradient domain and alpha matting, are widely used in the field of computer vision, and can typically achieve realistic results, especially for seamless boundaries. However, when the candidate composite images and the target images have obvious differences, such as color, texture and brightness, the composite results are unrealistic and inconsistent. At the same time, traditional compositing methods focus on basic feature matching, ignoring semantic rationality in composition processing. Quite a few compositing methods thus generate composite results without semantic rationality. In this paper, a new multi-scale image composition method has been presented. In the composition process, wavelet pyramid and basic feature handling were used to achieve multi-scale compositions. More importantly, a new criterion was established, based on the semantic rationality of images, which could ensure that the composite images are semantically valid. A large database was created to facilitate experimentation. The experiments showed that the methodology introduced in this paper produced superior results compared to traditional composition methods; the composite results were not only consistent and seamless, but were also semantically valid. [ABSTRACT FROM AUTHOR]

Image composition is a very important technique in computer generated imagery. Besides some factors such as contrast, texture and noise that affect the quality of the composition, color harmony between fore- and background is also an important factor that would affect the quality of the composition. However, in the previous image composition techniques, color harmony between fore- and background is seldom considered. In this paper, an optimization method is proposed to deal with the color harmonization problem that used in image composition. A cost function is derived from the local smoothness of the hue values, and the image is harmonized by minimizing the cost function. A new matching cost function is proposed to select the best matching harmonic schemes. Our approach overcomes several shortcomings of the existing color harmonization methods. We validate the performance of our method and demonstrate its effectiveness with a variety of experiments. [ABSTRACT FROM AUTHOR]

Abstract: In this work, mesoporous CeO2 crystalline film was prepared via calcining the solution of Ce(NO3)3 in ionic liquid (1-allyl-3-methylimidazolium chloride) containing cellulose. The as-prepared CeO2 film was characterized by means of different techniques including X-ray diffraction, transmission electron microscopy equipped with energy dispersive X-ray spectroscopy, scanning electron microscopy and nitrogen sorption analysis. It was indicated that the as-prepared CeO2 film was composed of numerous CeO2 crystals with size of ∼9nm, which assembled with each other to form interconnected channels within the film. The BET specific surface area and the pore volume of the CeO2 film were estimated to be 62m2/g and 0.15cm3/g, respectively, based on the N2 sorption isotherms. Being used as a support, the CeO2 film was loaded with Au nanoparticles of ∼5nm in size via the reduction of HAuCl4 with NaBH4 in aqueous solution. The as-synthesized Au/CeO2 nanocomposite demonstrated a good catalytic activity for aerobic oxidation of benzyl alcohol. [Copyright &y& Elsevier]

Abstract: Many real-world classification tasks involve discriminations between two unbalanced classes in imprecise environments, in which either the training data do not represent a random sample of the target population or the class distribution may shift over time in the target population. In such situations, in order to minimize the misclassification costs, the class distribution in target population must be known for selecting the optimal threshold. Forman has presented a method, based on the distribution generated on training data and the distribution on unlabeled test data, for estimating the number of positives in target population. However, when the data size is small, it is difficult to reliably generate these distributions for estimating the number of positives. This paper presents a novel algorithm to generate these distributions based on the bootstrap and Fisher discriminant analysis. Experiment results on five UCI data sets demonstrate its effectiveness. [Copyright &y& Elsevier]

Abstract: SnO2 nanofibers were obtained by electrospinning of a poly(vinyl alcohol) (PVA)/SnCl4·5H2O solution. A novel micro gas sensor based on the SnO2 nanofibers was successfully fabricated. The morphology and crystal structure of the fibers were analyzed by scanning electron microscope (SEM), transmission electron microscope (TEM) and X-ray diffraction (XRD), and the ethanol-sensing properties of the sensor were tested. The results showed that the SnO2 nanofibers with an average diameter of ∼100nm could be directly deposited on a microhotplate by “near-field” electrospinning. The micro gas sensor exhibited large response, low detection limit, fast response/recovery and good reproducibility. The detection limit was <10ppb and the response/recovery time towards 10ppm ethanol was <14s. [Copyright &y& Elsevier]

Abstract: A facile method to decorate the polymeric hollow spheres with ZnS nanoparticles has been presented. In this method, the precursors, Zn(Ac)2⋅H2O and CH3CSNH2, were first adsorbed by the polymer substrate in supercritical CO2–ethanol solution at 35 °C. Followed by heating the mixture at 100 °C for 2 h, ZnS/polymer composites were obtained. The as-produced ZnS/polymer composites were characterized by means of IR spectra, X-ray diffraction, transmission electron microscopy, high-resolution transmission electron microscopy. It was demonstrated that crystalline ZnS nanoparticles with size of 3–5 nm were uniformly decorated on the polymer spheres under suitable conditions. The resultant ZnS/polymer composites exhibited high efficiency for degrading eosin B, methyl orange and methylene blue under UV light irradiation. [Copyright &y& Elsevier]

Abstract: Metal oxide films, including Ce2O3 and/or CeO2, Al2O3, La2O3, were deposited on the outer surfaces of carbon nanotubes (CNTs) through the decomposition of metal nitrate precursors in supercritical CO2 modified with ethanol. Transmission electron microscopy showed that CNTs could be coated with metal oxide layers that were nominally complete and uniform. The thickness of the coating could be readily tailored by tuning the ratio of the initial mass of precursors to CNTs. The as-prepared CeO2–CNT composites showed high sensitivity and selectivity to acetone on the basis of chemiluminescence detection. [Copyright &y& Elsevier]

Abstract: Titanium dioxide (anatase, TiO2) nanoparticles have been successfully deposited onto multi-walled carbon nanotubes (MWCNTs) via hydrolysis of titanium isopropoxide in supercritical ethanol. The as-prepared composites were characterized by X-ray diffraction, transmission electron microscopy and X-ray photoelectron spectroscopy. It was demonstrated that the MWCNTs were decorated with well-dispersed anatase nanoparticles less than 7nm in diameter. The size and loading content of the nanoparticles on MWCNTs could be tuned by manipulating the ratio of precursor to MWCNTs, and the formation mechanism of the composites was also discussed. The absorbance spectrum of the resultant TiO2/MWCNT composites extended to the whole UV–visible region due to the decoration of TiO2 on MWCNTs. The TiO2/MWCNT composites were used as photocatalyst for phenol degradation under irradiation of visible light, which showed higher efficiency compared to a mixture of TiO2 and MWCNTs. [Copyright &y& Elsevier]

Abstract: PtRu nanoparticles were decorated on multi-walled carbon nanotubes (MWCNTs) using H2PtCl6 and RuCl3 as precursors with the aid of supercritical CO2, and the resulting composites were characterized by transmission electron microscopy (TEM), X-ray diffraction (XRD) and X-ray photoelectron spectroscopy. TEM observation showed that nanoparticles of size about 5nm were distributed evenly on the MWCNTs, and XRD analysis showed that the particles had a face-centered cubic crystal structure. The loading content of the nanoparticles on the MWCNTs could be adjusted by manipulating the relative ratio of the precursor to MWCNTs. The as-prepared PtRu/MWCNT composites exhibited high activity for methanol electro-oxidation. [Copyright &y& Elsevier]

Abstract: This work describes a method to decorate carbon nanotubes (CNTs) with metallic Pd and Ru nanocrystals via inorganic reactions in supercritical (SC) CO2–methanol solutions. In this route, PdCl2 or RuCl3⋅3H2O dissolved in SC CO2–methanol solution acted as a metal precursor and CNTs functioned as a template to direct the deposition of produced nanoparticles. Methanol served as the reductant for the precursors as well as cosolvent to enhance the dissolution of precursors in SC CO2. Dry products were readily obtained through in situ extraction with SC CO2 after reactions. The products were characterized by X-ray diffraction, X-ray photoelectron spectroscopy, and transmission electron microscopy. It was demonstrated that the loading content and particle size of the nanoparticles deposited on CNTs could be tuned by changing the weight ratio of the precursor to CNTs. This simple and efficient approach may also be utilized to synthesize other high-purity materials using inorganic salt precursors in SC CO2-based solution. [Copyright &y& Elsevier]

Abstract: Nanocomposites of zinc sulfide (ZnS) and montmorillonite (MMT) were prepared via a hydrothermal route. In this method, the MMT treated with hexadecyltrimethyl ammonium bromide (HTAB) aqueous solution was dispersed in the aqueous solution of thiourea and Zn(OOCHCH3)2⋅2H2O, and heated at 170 °C for about 4 h, resulting in ZnS-MMT composites. The as-prepared nanocomposites were characterized with X-ray diffraction, scanning electron microscopy, transmission electron microscopy, and nitrogen sorption analysis. It was demonstrated that the interlayer space of MMT was enlarged from 0.98 to 3.77 nm after the treatment with HTAB aqueous solution, and the ZnS nanoparticles were deposited on the layers of MMT. Nitrogen sorption analysis demonstrated that the specific surface area of the samples decreased from 39.2 m2/g of the pristine MMT to 5.9 m2/g of the final ZnS-MMT composites. The resulting ZnS-MMT nanocomposites (50.0 mg) could degrade eosin B completely in aqueous solution (75 ml, ) within 20 min under UV irradiation. [Copyright &y& Elsevier]

Abstract: A simple and efficient route to decorate carbon nanotubes (CNTs) with nanocrystalline RuO2 has been developed. In this method, RuCl3 ·3H2O was oxidized into RuO2 by oxygen in supercritical diethyl amine, and the produced RuO2 deposited on CNTs, resulting in RuO2/CNT nanocomposites. The as-prepared composites were structurally and morphologically characterized by X-ray diffraction, X-ray photoelectron spectroscopy, and transmission electron microscopy (TEM). TEM images showed that RuO2 nanoparticles attached on CNTs had uniform shape and a narrow particle size distribution. The loading content and the size of RuO2 particles on CNTs could be tuned by changing the mass ratio of RuCl3 ·3H2O/CNT. Electrochemical measurements by cyclic voltammetry demonstrated a substantial increment of the specific capacitance of CNTs due to a pseudocapacitance originated from the deposited RuO2 nanoparticles. [Copyright &y& Elsevier]

Abstract: Mesoporous aluminosilicate molecular sieves have been synthesized using K-feldspar, a natural ore rich in silicon and aluminum sources, as starting material. In this method, the mixture of K-feldspar and K2CO3 with mass ratio of 2:3 was first calcined at 880°C for 1.2h. The calcined materials were then dissolved in water together with cetyltrimethylammonium bromide as structure-directing agent, resulting in mesoporous molecular sieves after heating the solution at 130°C for 60h. Scanning electron microscopy and transmission electron microscopy observations indicated that the resulting materials were spherical particles with size of about 100nm. The mesoporous structure of the as-synthesized materials was confirmed by low angle X-ray diffraction and nitrogen sorption analysis. The BET surface area of the as-prepared material after calcined at 550°C was 507m2 g−1 and the pore volume was 0.854cm3 g−1. 27Al MAS NMR analysis showed that nearly all of aluminium is incorporated into the framework of the mesoporous material with 4-coordinated state. The acidity of the product was analyzed by FTIR spectroscopy of pyridine-adsorbed product. The product was treated in water at 100°C for 360h, and its mesoporous structure was still intact, suggesting its high hydrothermal stability. [Copyright &y& Elsevier]

Homogeneity fiber/matrix composites of isotactic polypropylene (iPP) were prepared with both partially carbon-coated and non-carbon-coated iPP fibers. Their morphologies produced by melt recrystallization were studied by means of polarized optical microscopy. The results show that through vacuum evaporating a thin carbon film partially on the surface of iPP fiber, the nucleation ability of the molten iPP matrix during the course of recooling has been enhanced tremendously. The early formation and high density of the iPP row nuclei formed along the partially carbon-coated iPP fibers lead to the formation of an apparent iPP transcrystalline zone in the vicinity of its precoated fiber. The high nucleation ability of the carbon-coated iPP fiber towards its homogeneity matrix may originate from the surface fixing effect of the vacuum evaporated carbon layer on the polymer samples. [Copyright &y& Elsevier]