Search

In this work, the properties of a two-tone signal driven concurrent power amplifier (CPA) are analyzed extensively. Firstly, the characteristics of two-tone signals are discussed to explain the nonlinearity and efficiency of two-tone signal driven CPA. Secondly, a method of how to solve the efficiency of a two-tone signal driven CPA is proposed in this paper with a detailed calculation procedure and complete theory. Finally, a general empirical expression to calculate the drain efficiency (DE) of CPA versus the frequency ratio and the amplitude ratio of the two carriers is proposed. Simulation results and experimental verifications are given to validate the proposed analytical formulation to predict the efficiency of CPAs when driven with balanced or imbalanced two-tone signals.

Differentiable architecture search (DARTS) transforms architectural optimization into a super network optimization by stacking two cells (2 c.). However, repeatedly stacking two cells is a suboptimal operation since cells in different depths should be various. Besides, we find that the performance is slightly improved by increasing the number of searched cells (e.g., from 2 c. to 5 c.), but it leads to uneven resource allocation. This letter proposes a regularized DARTS (RDARTS) to adjust the architectural differences and balance degrees of freedom and resource allocation. Specifically, we use separate architectural parameters for two reduction cells and three normal cells, and then propose an Reg distance to calculate the difference between cells. We design a new validation loss which is the weighting of cross-entropy and Reg loss and introduce an adaptive adjustment method. Results show that RDARTS achieves the top-1 accuracy of 97.64% and 75.8% on CIFAR and ImageNet. [ABSTRACT FROM AUTHOR]

Rapidly developing Redirected Walking (ROW) technologies have enabled VR applications to immerse users in large virtual environments (VE) while actually walking in relatively small physical environments (PE). When an unavoidable collision emerges in a PE, the ROW controller suspends the user's immersive experience and resets the user to a new direction in PE. Existing ROW methods mainly aim to reduce the number of resets. However, from the perspective of the user experience, when users are about to reach a point of interest (POI) in a VE, reset interruptions are more likely to have an impact on user experience. In this paper, we propose a new ROW method, aiming to keep resets occurring at a longer distance from the virtual target, as well as to reduce the number of resets. Simulation experiments and real user studies demonstrate that our method outperforms state-of-the-art ROW methods in the number of resets and dramatically increases the distance between the reset locations and the virtual targets. [ABSTRACT FROM AUTHOR]

Conditional generative adversarial networks (cGANs) are designed to generate images based on the provided conditions, e.g., class-level distributions, semantic label maps, etc. Existing methods have used the same generator architecture for all classes. This paper presents an idea that adopts neural architecture search (NAS) to find a class-aware architecture for each class. The search space contains regular and class-modulated convolutions, where the latter is designed to introduce class-specific information while avoiding the reduction of training data for each class generator. The search algorithm follows a weight-sharing pipeline with mixed-architecture optimization so that the search cost does not grow with the number of classes. To learn the sampling policy, a Markov decision process is embedded into the search algorithm, and a moving average is applied for better stability. Class-aware generators show advantages over class-agnostic architectures experimentally. Moreover, we discover two intriguing phenomena that are inspirational to craft cGANs by hand. [ABSTRACT FROM AUTHOR]

Person re-identification (re-ID) in the scenario with large spatial and temporal spans has not been fully explored. This fact partially occurs because existing benchmark datasets were mainly collected with limited spatial and temporal ranges, e.g., using videos recorded in a few days by cameras in a specific region of the campus. Such limited spatial and temporal ranges make it hard to simulate the difficulties of person re-ID in real scenarios. In this work, we contribute a novel Large-scale Spatio-Temporal (LaST) person re-ID dataset, including 10,862 identities with more than 228k images. Compared with existing datasets, LaST presents more challenging and high-diversity re-ID settings and significantly larger spatial and temporal ranges. For instance, each person can appear in different cities or countries, and in various time slots from day to evening, and in different seasons from spring to winter. To our best knowledge, LaST is a novel person re-ID dataset with the largest spatio-temporal ranges. Based on LaST, we verified its challenge by conducting a comprehensive performance evaluation of 14 re-ID algorithms. We further propose an easy-to-implement baseline that works well in such challenging re-ID settings. We also verified that models pre-trained on LaST can generalize well on existing datasets with short-term and cloth-changing scenarios. We expect LaST to inspire future works toward more realistic and challenging re-ID tasks. More information about the dataset is available at https://github.com/shuxjweb/last.git. [ABSTRACT FROM AUTHOR]

Visible-infrared person re-identification (VI-ReID) is challenging due to the large modality discrepancy between visible and infrared images. Existing methods mainly focus on learning modality-shared representations by embedding images from different modalities into a common feature space, in which some discriminative modality information is discarded. Different from these methods, in this paper, we propose a novel Modality-Specific Memory Network (MSMNet) to complete the missing modality information and aggregate visible and infrared modality features into a unified feature space for the VI-ReID task. The proposed model enjoys several merits. First, it can exploit the missing modality information to alleviate the modality discrepancy when only the single-modality input is provided. To the best of our knowledge, this is the first work to exploit the missing modality information completion and alleviate the modality discrepancy with the memory network. Second, to guide the learning process of the memory network, we design three effective learning strategies, including feature consistency, memory representativeness and structural alignment. By incorporating these learning strategies in a unified model, the memory network can be well learned to propagate identity-related information between modalities and boost the VI-ReID performance. Extensive experimental results on two standard benchmarks (SYSU-MM01 and RegDB) demonstrate that the proposed MSMNet performs favorably against state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Recent studies have pointed out that many well-developed Visual Question Answering (VQA) models are heavily affected by the language prior problem. It refers to making predictions based on the co-occurrence pattern between textual questions and answers instead of reasoning upon visual contents. To tackle this problem, most existing methods focus on strengthening the visual feature learning capability to reduce this text shortcut influence on model decisions. However, few efforts have been devoted to analyzing its inherent cause and providing an explicit interpretation. It thus lacks a good guidance for the research community to move forward in a purposeful way, resulting in model construction perplexity towards overcoming this non-trivial problem. In this paper, we propose to interpret the language prior problem in VQA from a class-imbalance view. Concretely, we design a novel interpretation scheme whereby the loss of mis-predicted frequent and sparse answers from the same question type is distinctly exhibited during the late training phase. It explicitly reveals why the VQA model tends to produce a frequent yet obviously wrong answer, to a given question whose right answer is sparse in the training set. Based upon this observation, we further propose a novel loss re-scaling approach to assign different weights to each answer according to the training data statistics for estimating the final loss. We apply our approach into six strong baselines and the experimental results on two VQA-CP benchmark datasets evidently demonstrate its effectiveness. In addition, we also justify the validity of the class imbalance interpretation scheme on other computer vision tasks, such as face recognition and image classification. [ABSTRACT FROM AUTHOR]

In order to overcome the demerits of traditional city air terminals (CAT), a brand-new service mode based on urban mobile stations (UMS) is introduced in this article for providing the luggage check-in service in urban area. Different from CAT, for each service time period, the UMSs are located based on the real-time passenger distributions. The objectives are to improve the service quality and enhance the competitiveness of air transportation. First, the system components and operation mode of UMS are described. Second, a mathematical model is set up for locating UMSs and assigning passengers to service stations. Three aspects, including the average path length from passengers to UMSs, the maximum acceptable path length for passengers, and the maximum service capacity of a station are considered. Third, to meet the strict time requirement of the UMS system, an effective evolutionary algorithm for optimizing UMS locations in real time is further developed. In the method, the ripple-spreading algorithm is applied for solving the many-to-many path optimization problem and a two-stage adaptive genetic algorithm is utilised for efficiently locating UMSs. Finally, a test case is set up based on the city centre of Tianjin, China, and passenger source locations are obtained by surveying passengers at Tianjin Airport. The reported method is then applied to dynamically locate UMSs in Tianjin urban area and it is compared to some other existing methods to show its effectiveness and efficiency. The service quality of UMS is compared to CAT in both cases with realistic data and randomly generated data to show its advantages. [ABSTRACT FROM AUTHOR]

Although person re-identification (person re-id) has advanced substantially in recent years, most methods are based on the assumption that the identities would not change clothes. This assumption might not hold in practice considering criminals who intentionally change clothes. In this work, we attempt to solve person re-id under moderate clothing change. Since the human body shape is considered as relatively more invariant under moderate clothing changes, we propose to learn a reliable shape-aware feature representation by mutually learning both colorful images and contour images. Instead of directly extracting shape features from contour images, we utilize contour feature learning as regularization and excavate more effective shape-aware feature representations from colorful images. We propose a multi-scale appearance and contour deep infomax (MAC-DIM) to maximize mutual information between colorful appearance features and contour shape features, and in this way, the extracted appearance features are constrained to be shape-aware in terms of both low-level visual properties and high-level semantics. To better model the long-range human body shape and explicitly capture contour segment relations, we introduce hierarchical graph modeling as aggregation headers, propagating structural context through graph convolutional networks (GCNs). The extensive results on benchmarks under clothing changes demonstrate the effectiveness of our shape-aware feature learning scheme. [ABSTRACT FROM AUTHOR]

3D skeleton-based action recognition and motion prediction are two essential problems of human activity understanding. In many previous works: 1) they studied two tasks separately, neglecting internal correlations; and 2) they did not capture sufficient relations inside the body. To address these issues, we propose a symbiotic model to handle two tasks jointly; and we propose two scales of graphs to explicitly capture relations among body-joints and body-parts. Together, we propose symbiotic graph neural networks, which contain a backbone, an action-recognition head, and a motion-prediction head. Two heads are trained jointly and enhance each other. For the backbone, we propose multi-branch multiscale graph convolution networks to extract spatial and temporal features. The multiscale graph convolution networks are based on joint-scale and part-scale graphs. The joint-scale graphs contain actional graphs, capturing action-based relations, and structural graphs, capturing physical constraints. The part-scale graphs integrate body-joints to form specific parts, representing high-level relations. Moreover, dual bone-based graphs and networks are proposed to learn complementary features. We conduct extensive experiments for skeleton-based action recognition and motion prediction with four datasets, NTU-RGB+D, Kinetics, Human3.6M, and CMU Mocap. Experiments show that our symbiotic graph neural networks achieve better performances on both tasks compared to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Face aging has attracted widespread attention in recent years, but most studies are based on the same emotional situation. Is the same person’s aging in different emotional situations the same? To solve the above confusion, this paper proposes a novel face aging model DEF-Net, which consists of two parts: different emotional learnings (Emotion-Net) and face aging (Age-Net). Given a target emotion category, DEF-Net first assists the image from the original dataset to learn the emotion features through Emotion-Net and the generated dataset is used as the inputs of Age-Net. At the same time, multiple loss functions are used to ensure that the crucial information of the original image is not lost. Secondly, Age-Net, which has been pre-trained on the original dataset, began to adopt the generated dataset to learn the aging distribution under different emotions. Designed loss functions are utilized to ensure that the realistic target images generated by Age-Net do not lose the learned emotional characteristics. Finally, extensive experiments are used to verify the performance of DEF-Net. Compared with other state-of-the-art methods: (1) DEF-Net can learn different facial emotions across different datasets and generate corresponding realistic aging images; (2) the results achieved by our DEF-Net are demonstrated to be better than those by the model that performs face aging first and then learns different emotional characteristics. [ABSTRACT FROM AUTHOR]

As an emerging field of network content production, live video has been in the vacuum zone of cyberspace governance for a long time. Streamer action recognition is conducive to the supervision of live video content. In view of the diversity and imbalance of streamer actions, it is attractive to introduce few-shot learning to realize streamer action recognition. Therefore, a meta-learning paradigm and CosAttn for streamer action recognition method in live video is proposed, including: (1) the training set samples similar to the streamer action to be recognized are pretrained to improve the backbone network; (2) video-level features are extracted by R(2+1)D-18 backbone and global average pooling in the meta-learning paradigm; (3) the streamer action is recognized by calculating cosine similarity after sending the video-level features to CosAttn to generate a streamer action category prototype. Experimental results on several real-world action recognition datasets demonstrate the effectiveness of our method. [ABSTRACT FROM AUTHOR]

Pedestrian detection in the wild remains a challenging problem especially for scenes containing serious occlusion. In this paper, we propose a novel feature learning method in the deep learning framework, referred to as Feature Calibration Network (FC-Net), to adaptively detect pedestrians under various occlusions. FC-Net is based on the observation that the visible parts of pedestrians are selective and decisive for detection, and is implemented as a self-paced feature learning framework with a self-activation (SA) module and a feature calibration (FC) module. In a new self-activated manner, FC-Net learns features which highlight the visible parts and suppress the occluded parts of pedestrians. The SA module estimates pedestrian activation maps by reusing classifier weights, without any additional parameter involved, therefore resulting in an extremely parsimony model to reinforce the semantics of features, while the FC module calibrates the convolutional features for adaptive pedestrian representation in both pixel-wise and region-based ways. Experiments on CityPersons and Caltech datasets demonstrate that FC-Net improves detection performance on occluded pedestrians up to 10% while maintaining excellent performance on non-occluded instances. [ABSTRACT FROM AUTHOR]

A low-profile double U-slot microstrip patch antenna with a wide operating bandwidth has been developed in this article. This design produces the dual radiative resonant modes adjacent to each even at the thickness of the antenna of $0.02 {\lambda _{0}}$. An extremely stable radiation gain that varied from 6.2 to 6.8 dBi is achieved within the whole operating band. In addition, detailed parametric studies in this article provide antenna engineers with simple approximate rules that result in an excellent first-pass design with prescribed characteristics requiring only minimal tuning. Finally, the proposed antenna with the thickness of $0.04 {\lambda _{0}}$ is fabricated and measured. Simulated and measured results are found in good agreement with each other. They have evidently illustrated that the bandwidth is increased by 50% compared to the traditional U-slot antenna. With the help of these simple design procedures and examples in this article, antenna designers could easily develop a low-profile broadband antenna. [ABSTRACT FROM AUTHOR]

Recently, emotion recognition in real sceneries such as in the wild has attracted extensive attention in affective computing, because existing spontaneous emotions in real sceneries are more challenging and difficult to identify than other emotions. Motivated by the diverse effects of different lengths of audio spectrograms on emotion identification, this paper proposes a multiscale deep convolutional long short-term memory (LSTM) framework for spontaneous speech emotion recognition. Initially, a deep convolutional neural network (CNN) model is used to learn deep segment-level features on the basis of the created image-like three channels of spectrograms. Then, a deep LSTM model is adopted on the basis of the learned segment-level CNN features to capture the temporal dependency among all divided segments in an utterance for utterance-level emotion recognition. Finally, different emotion recognition results, obtained by combining CNN with LSTM at multiple lengths of segment-level spectrograms, are integrated by using a score-level fusion strategy. Experimental results on two challenging spontaneous emotional datasets, i.e., the AFEW5.0 and BAUM-1s databases, demonstrate the promising performance of the proposed method, outperforming state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Real-time semantic video segmentation is a challenging task due to the strict requirements of inference speed. Recent approaches mainly devote great efforts to reducing the model size for high efficiency. In this paper, we rethink this problem from a different viewpoint: using knowledge contained in compressed videos. We propose a simple and effective framework, dubbed TapLab, to tap into resources from the compressed domain. Specifically, we design a fast feature warping module using motion vectors for acceleration. To reduce the noise introduced by motion vectors, we design a residual-guided correction module and a residual-guided frame selection module using residuals. TapLab significantly reduces redundant computations of the state-of-the-art fast semantic image segmentation models, running 3 to 10 times faster with controllable accuracy degradation. The experimental results show that TapLab achieves 70.6 percent mIoU on the Cityscapes dataset at 99.8 FPS with a single GPU card for the 1024×2048 videos. A high-speed version even reaches the speed of 160+ FPS. Code will be available soon at https://github.com/Sixkplus/TapLab. [ABSTRACT FROM AUTHOR]

Learning to generate continuous linguistic descriptions for multi-subject interactive videos in great details has particular applications in team sports auto-narrative. In contrast to traditional video caption, this task is more challenging as it requires simultaneous modeling of fine-grained individual actions, uncovering of spatio-temporal dependency structures of frequent group interactions, and then accurate mapping of these complex interaction details into long and detailed commentary. To explicitly address these challenges, we propose a novel framework Graph-based Learning for Multi-Granularity Interaction Representation (GLMGIR) for fine-grained team sports auto-narrative task. A multi-granular interaction modeling module is proposed to extract among-subjects’ interactive actions in a progressive way for encoding both intra- and inter-team interactions. Based on the above multi-granular representations, a multi-granular attention module is developed to consider action/event descriptions of multiple spatio-temporal resolutions. Both modules are integrated seamlessly and work in a collaborative way to generate the final narrative. In the meantime, to facilitate reproducible research, we collect a new video dataset from YouTube.com called Sports Video Narrative dataset (SVN). It is a novel direction as it contains $6K$ 6 K team sports videos (i.e., NBA basketball games) with $10K$ 10 K ground-truth narratives(e.g., sentences). Furthermore, as previous metrics such as METEOR (i.e., used in coarse-grained video caption task) DO NOT cope with fine-grained sports narrative task well, we hence develop a novel evaluation metric named Fine-grained Captioning Evaluation (FCE), which measures how accurate the generated linguistic description reflects fine-grained action details as well as the overall spatio-temporal interactional structure. Extensive experiments on our SVN dataset have demonstrated the effectiveness of the proposed framework for fine-grained team sports video auto-narrative. [ABSTRACT FROM AUTHOR]

The large pose variations and misalignment errors exhibited by person images significantly increase the difficulty of person Re-Identification (ReID). Existing works commonly apply extra operations like pose estimation, part segmentation, etc., to alleviate those issues and improve the robustness of pedestrian representations. While boosting the ReID accuracy, those operations introduce considerable computational overheads and make the deep models complex and hard to tune. To chase a more efficient solution, we propose a Part-Guided Representation (PGR) composed of Pose Invariant Feature (PIF) and Local Descriptive Feature (LDF), respectively. We call PGR “Part-Guided” because it is trained and supervised by local part cues. Specifically, PIF approximates a pose invariant representation inferred by pose estimation and pose normalization. LDF focuses on discriminative body parts by approximating a representation learned with body region segmentation. In this way, extra pose extraction is only introduced during the training stage to supervise the learning of PGR, but is not required during the testing stage for feature extraction. Extensive comparisons with recent works on five widely used datasets demonstrate the competitive accuracy and efficiency of PGR. [ABSTRACT FROM AUTHOR]

Person re-identification (ReID) aims at matching identities across disjoint cameras. Its fundamental difficulty lies in associating images across individual cameras, where a key clue, i.e., identity appearance, is prone to the environmental factors of cameras and, consequently, subject to distinct image distributions due to the environmental differences between cameras. To associate images from training cameras, ReID methods strongly demand expensive inter-camera annotations for learning the relations between the distribution of these cameras, yet trained models are still not guaranteed to transfer well to unseen cameras. This problem significantly limits the application of ReID. This paper rethinks the working mechanism of conventional ReID approaches and puts forward a new solution. With an effective operator named Camera-based Batch Normalization (CBN), we guarantee an invariant input distribution independent of all cameras. Thus, the training and testing procedures are always conducted under the same input distribution. This alignment brings three benefits. First, ReID models enjoy better abilities to generalize across testing scenarios with unseen cameras and transfer across multiple training sets. Second, it makes better use of intra-camera annotations, which have been undervalued before due to the lack of cross-camera information. Ideally, the cost of inter-camera annotations can be largely reduced. Third, cross-modality tasks can be better defined through aligning visible/infrared cameras’ distributions. Experiments on a wide range of ReID tasks demonstrate the effectiveness of our approach. [ABSTRACT FROM AUTHOR]

Many pioneering approaches have verified the effectiveness of utilizing the global temporal and local object information for video understanding tasks and have achieved significant progress. However, existing methods utilize object detectors to extract all objects overall video frames. This may bring performance degradation due to the information redundancy both spatially and temporally. To address this problem, we propose an adaptive spatial location module for the video captioning task which dynamically predicts an important position of each video frame in the procedure of generating the description sentence. The proposed adaptive spatial location method not only makes our model focus on local object information, but also reduces time and memory consumption brought by the temporal redundancy in extensive video frames and improves the accuracy of generated description. Besides, we propose a balanced loss function to address the class imbalance problem existing in training data. The proposed balanced loss assigns different weight to each word of ground-truth sentence in the training process which can generate more diversified description sentences. Extensive experimental results on the MSVD and MSR-VTT dataset show that the proposed method achieves competitive performance compared to state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Temporal action localization, detecting actions in untrimmed videos, is widely studied by anchor-based approaches that first generate excessive action proposals, i.e., temporal windows, then evaluate and classify these proposals. To reduce the number of action proposals, recent studies use an anchor-free approach that leverages each time point rather than a temporal window to represent an action instance. However, this point representation, usually modeled by temporal convolutions, may have the fixed and limited receptive field to detect an entire action. So we propose an Actionness-guided Transformer (Ag-Trans) model to learn representations for each point proposal. Ag-Trans first predicts the actionness, i.e., time sequences of the action starting, continuing, and ending phases, then the corresponding action phase can be embedded to model the point representation. Experimental results show that the Ag-Trans model outperforms the CNN-based model under the same experiment settings, especially for long-duration actions. [ABSTRACT FROM AUTHOR]

The welding assemblies are the main components of the platform screen door system, which can offer the subway with safety, economy, and silence. The deformed welding assemblies can reduce the carrying capacity of the platform screen door system, so the flatness needs to be measured before installation. In this study, a flatness measuring robot based on stereo vision and grid pattern projector measuring system is developed to online measure the welding assembly flatness in workshop. The developed robot is capable of measuring the flatness by incorporating the following solutions: 1) measuring system consisting of stereo vision and grid pattern projector is designed; 2) grid intersection location method using grid intersection probability calculation, candidate preselection, grid intersection reexamination and sub-pixel refinement is proposed, which can consider both false positives and false negatives by using preselection-and-reexamination; 3) 3D coordinates of sampling points (grid intersection) are calculated based on stereo camera calibration and robot hand-eye calibration; 4) flatness measuring method based on the least squares plan is employed to calculate the flatness of welding assembly. The developed flatness measuring robot is robust, accurate, and efficient, as demonstrated by the following performances: an average measuring bias of 0.10 mm in actual working conditions, and an average measurement time of 4.1 s for single welding assembly, meeting the engineering requirements. It is foreseeable that the developed flatness measuring robot has great application market in the online measurement with a middling accuracy and high efficiency. [ABSTRACT FROM AUTHOR]

Differentiable architecture search (DARTS) enables effective neural architecture search (NAS) using gradient descent, but suffers from high memory and computational costs. In this paper, we propose a novel approach, namely Partially-Connected DARTS (PC-DARTS), to achieve efficient and stable neural architecture search by reducing the channel and spatial redundancies of the super-network. In the channel level, partial channel connection is presented to randomly sample a small subset of channels for operation selection to accelerate the search process and suppress the over-fitting of the super-network. Side operation is introduced for bypassing (non-sampled) channels to guarantee the performance of searched architectures under extremely low sampling rates. In the spatial level, input features are down-sampled to eliminate spatial redundancy and enhance the efficiency of the mixed computation for operation selection. Furthermore, edge normalization is developed to maintain the consistency of edge selection based on channel sampling with the architectural parameters for edges. Theoretical analysis shows that partial channel connection and parameterized side operation are equivalent to regularizing the super-network on the weights and architectural parameters during bilevel optimization. Experimental results demonstrate that the proposed approach achieves higher search speed and training stability than DARTS. PC-DARTS obtains a top-1 error rate of 2.55 percent on CIFAR-10 with 0.07 GPU-days for architecture search, and a state-of-the-art top-1 error rate of 24.1 percent on ImageNet (under the mobile setting) within 2.8 GPU-days. [ABSTRACT FROM AUTHOR]

We propose a multiscale spatio-temporal graph neural network (MST-GNN) to predict the future 3D skeleton-based human poses in an action-category-agnostic manner. The core of MST-GNN is a multiscale spatio-temporal graph that explicitly models the relations in motions at various spatial and temporal scales. Different from many previous hierarchical structures, our multiscale spatio-temporal graph is built in a data-adaptive fashion, which captures nonphysical, yet motion-based relations. The key module of MST-GNN is a multiscale spatio-temporal graph computational unit (MST-GCU) based on the trainable graph structure. MST-GCU embeds underlying features at individual scales and then fuses features across scales to obtain a comprehensive representation. The overall architecture of MST-GNN follows an encoder-decoder framework, where the encoder consists of a sequence of MST-GCUs to learn the spatial and temporal features of motions, and the decoder uses a graph-based attention gate recurrent unit (GA-GRU) to generate future poses. Extensive experiments are conducted to show that the proposed MST-GNN outperforms state-of-the-art methods in both short and long-term motion prediction on the datasets of Human 3.6M, CMU Mocap and 3DPW, where MST-GNN outperforms previous works by 5.33% and 3.67% of mean angle errors in average for short-term and long-term prediction on Human 3.6M, and by 11.84% and 4.71% of mean angle errors for short-term and long-term prediction on CMU Mocap, and by 1.13% of mean angle errors on 3DPW in average, respectively. We further investigate the learned multiscale graphs for interpretability. [ABSTRACT FROM AUTHOR]

Conversational image search, a revolutionary search mode, is able to interactively induce the user response to clarify their intents step by step. Several efforts have been dedicated to the conversation part, namely automatically asking the right question at the right time for user preference elicitation, while few studies focus on the image search part given the well-prepared conversational query. In this paper, we work towards conversational image search, which is much difficult compared to the traditional image search task, due to the following challenges: 1) understanding complex user intents from a multimodal conversational query; 2) utilizing multiform knowledge associated images from a memory network; and 3) enhancing the image representation with distilled knowledge. To address these problems, in this paper, we present a novel contextuaL imAge seaRch sCHeme (LARCH for short), consisting of three components. In the first component, we design a multimodal hierarchical graph-based neural network, which learns the conversational query embedding for better user intent understanding. As to the second one, we devise a multi-form knowledge embedding memory network to unify heterogeneous knowledge structures into a homogeneous base that greatly facilitates relevant knowledge retrieval. In the third component, we learn the knowledge-enhanced image representation via a novel gated neural network, which selects the useful knowledge from retrieved relevant one. Extensive experiments have shown that our LARCH yields significant performance over an extended benchmark dataset. As a side contribution, we have released the data, codes, and parameter settings to facilitate other researchers in the conversational image search community. [ABSTRACT FROM AUTHOR]

Weakly supervised temporal action detection has better scalability and practicability than fully supervised action detection in reality deployment. However, it is difficult to learn a robust model without temporal action boundary annotations. In this paper, we propose an en-to-end Multi-Scale Structure-Aware Network (MSA-Net) for weakly supervised temporal action detection by exploring both the global structure information of a video and the local structure information of actions. The proposed SA-Net enjoys several merits. First, to localize actions with different durations, each video is encoded into feature representations with different temporal scales. Second, based on the multi-scale feature representation, the proposed model has designed two effective structure modeling mechanisms including global structure modeling and local structure modeling, which can effectively learn discriminative structure aware representations for robust and complete action detection. To the best of our knowledge, this is the first work to fully explore the global and local structure information in a unified deep model for weakly supervised action detection. And extensive experimental results on two benchmark datasets demonstrate that the proposed MSA-Net performs favorably against state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Gait recognition aims to recognize persons’ identities by walking styles. Gait recognition has unique advantages due to its characteristics of non-contact and long-distance compared with face and fingerprint recognition. Cross-view gait recognition is a challenge task because view variance may produce large impact on gait silhouettes. The development of deep learning has promoted cross-view gait recognition performances to a higher level. However, performances of existing deep learning-based cross-view gait recognition methods are limited by lack of gait samples under different views. In this paper, we take a Multi-view Gait Generative Adversarial Network (MvGGAN) to generate fake gait samples to extend existing gait datasets, which provides adequate gait samples for deep learning-based cross-view gait recognition methods. The proposed MvGGAN method trains a single generator for all view pairs involved in single or multiple datasets. Moreover, we perform domain alignment based on projected maximum mean discrepancy to reduce the influence of distribution divergence caused by sample generation. The experimental results on CASIA-B and OUMVLP dataset demonstrate that fake gait samples generated by the proposed MvGGAN method can improve performances of existing state-of-the-art cross-view gait recognition methods obviously on both single-dataset and cross-dataset evaluation settings. [ABSTRACT FROM AUTHOR]

Natural language moment localization aims at localizing video clips according to a natural language description. The key to this challenging task lies in modeling the relationship between verbal descriptions and visual contents. Existing approaches often sample a number of clips from the video, and individually determine how each of them is related to the query sentence. However, this strategy can fail dramatically, in particular when the query sentence refers to some visual elements that appear outside of, or even are distant from, the target clip. In this paper, we address this issue by designing an Interaction-Integrated Network (I2N), which contains a few Interaction-Integrated Cells (I2Cs). The idea lies in the observation that the query sentence not only provides a description to the video clip, but also contains semantic cues on the structure of the entire video. Based on this, I2Cs go one step beyond modeling short-term contexts in the time domain by encoding long-term video content into every frame feature. By stacking a few I2Cs, the obtained network, I2N, enjoys an improved ability of inference, brought by both (I) multi-level correspondence between vision and language and (II) more accurate cross-modal alignment. When evaluated on a challenging video moment localization dataset named DiDeMo, I2N outperforms the state-of-the-art approach by a clear margin of 1.98%. On other two challenging datasets, Charades-STA and TACoS, I2N also reports competitive performance. [ABSTRACT FROM AUTHOR]

Person re-identification is a crucial task of identifying pedestrians of interest across multiple surveillance camera views. For person re-identification, a pedestrian is usually represented with features extracted from a rectangular image region that inevitably contains the scene background, which incurs ambiguity to distinguish different pedestrians and degrades the accuracy. Thus, we propose an end-to-end foreground-aware network to discriminate against the foreground from the background by learning a soft mask for person re-identification. In our method, in addition to the pedestrian ID as supervision for the foreground, we introduce the camera ID of each pedestrian image for background modeling. The foreground branch and the background branch are optimized collaboratively. By presenting a target attention loss, the pedestrian features extracted from the foreground branch become more insensitive to backgrounds, which greatly reduces the negative impact of changing backgrounds on pedestrian matching across different camera views. Notably, in contrast to existing methods, our approach does not require an additional dataset to train a human landmark detector or a segmentation model for locating the background regions. The experimental results conducted on three challenging datasets, i.e., Market-1501, DukeMTMC-reID, and MSMT17, demonstrate the effectiveness of our approach. [ABSTRACT FROM AUTHOR]

Cross-modal retrieval aims to identify relevant data across different modalities. In this work, we are dedicated to cross-modal retrieval between images and text sentences, which is formulated into similarity measurement for each image-text pair. To this end, we propose a Cross-modal Relation Guided Network (CRGN) to embed image and text into a latent feature space. The CRGN model uses GRU to extract text feature and ResNet model to learn the globally guided image feature. Based on the global feature guiding and sentence generation learning, the relation between image regions can be modeled. The final image embedding is generated by a relation embedding module with an attention mechanism. With the image embeddings and text embeddings, we conduct cross-modal retrieval based on the cosine similarity. The learned embedding space well captures the inherent relevance between image and text. We evaluate our approach with extensive experiments on two public benchmark datasets, i.e., MS-COCO and Flickr30K. Experimental results demonstrate that our approach achieves better or comparable performance with the state-of-the-art methods with notable efficiency. [ABSTRACT FROM AUTHOR]

Visual can be easily disturbed by similar surrounding objects. Such objects as hard distractors, even though being the minority among negative samples, increase the risk of target drift and model corruption, which deserve additional attention in online tracking and model update. To enhance the tracking robustness, in this paper, we propose a cascaded regression tracker with two sequential stages. In the first stage, we filter out abundant easily-identified negative candidates via an efficient convolutional regression. In the second stage, a discrete sampling based ridge regression is designed to double-check the remaining ambiguous hard samples, which serves as an alternative of fully-connected layers and benefits from the closed-form solver for efficient learning. During the model update, we utilize the hard negative mining technique and an adaptive ridge regression scheme to improve the discrimination capability of the second-stage regressor. Extensive experiments are conducted on 11 challenging tracking benchmarks including OTB-2013, OTB-2015, VOT2018, VOT2019, UAV123, Temple-Color, NfS, TrackingNet, LaSOT, UAV20L, and OxUvA. The proposed method achieves state-of-the-art performance on prevalent benchmarks, while running in a real-time speed. [ABSTRACT FROM AUTHOR]

Long-term Video Question Answering plays an essential role in visual information retrieval, which aims at generating natural language answers to discretionary free-form questions about the referenced long-term video. Rather than remember the video as a sequence of visual content, humans have an innate cognitive ability to identify the critical moments related to the question at first glance, then tie together the specific evidence around these critical moments for further analysis and reasoning. Motivated by this intuition, we propose the multimodal hierarchical memory attentive networks with two heterogeneous memory subnetworks: the top guided memory network and the bottom enhanced multimodal memory attentive network. The top guided memory network serves as a shallow inference engine to pick relevant and informative moments of questions and obtain salient video content at a coarse-grained level. Subsequently, the bottom enhanced multimodal memory attentive network is designed as an in-depth reasoning engine to perform more accurate attention with cues from video bottom evidence in a fine-grained level to enhance question answering quality. We evaluate the proposed method on three publicly available video question answering benchmarks, namely ActivityNet-QA, MSRVTT-QA, and MSVD-QA. Experimental results demonstrate that the proposed approach significantly outperforms other state-of-the-art methods for long-term videos. Extensive ablation studies are carried out to explore the reasons behind the proposed model’s effectiveness. [ABSTRACT FROM AUTHOR]

Live video streaming platforms have attracted millions of streamers and daily active users. For profit and popularity accumulation, some streamers mix pornography content into live content to avoid online supervision. Therefore, accurate recognition of porn streamers in live video streaming has become a challenging task. Porn streamers in live video present multimodal characteristics including visual and acoustic content. Therefore, a porn streamer recognition method in live video streaming is proposed that uses attention-gated multimodal deep features. Our contribution includes the following: (1) multimodal deep features, i.e., spatial, motion and audio, are extracted from live video streaming using convolutional neural networks (CNNs), in which the temporal context of multimodal features is obtained with a bi-directional gated recurrent unit (Bi-GRU); (2) the tri-attention gated mechanism is applied to map the associations between different modalities by assigning higher weights to important features for further reduction in the redundancy of multimodal features; (3) porn streamers in live video streaming are recognized via the attention-gated multimodal deep features. Six experiments are conducted on a real-world dataset, and the competitive results demonstrate that our method can effectively recognize porn streamers in live video streaming. [ABSTRACT FROM AUTHOR]

Cross-media hashing, which maps data from different modalities to a low-dimensional sharing Hamming space, has attracted considerable attention due to the rapid increase of multimodal data, for example, images and texts. Recent cross-media hashing works mainly aim at learning compact hash codes to preserve the class label-based or feature-based similarities among samples. However, these methods ignore the unbalanced semantic gaps between different modalities and high-level semantic concepts, which generally results in less effective hash functions and unsatisfying retrieval performance. Specifically, the key words of texts contain semantic meanings, while the low-level features of images lack of semantic meanings. That means the semantic gap in image modality is larger than that in text modality. In this paper, we propose a simple yet effective hashing method for cross-media retrieval to address this problem, dubbed discrete semantic alignment hashing (DSAH). First, DSAH formulates to exploit collaborative filtering to mine the relations between class labels and hash codes, which can reduce memory consumption and computational cost compared to pairwise similarity. Then, the attribute of image modality is employed to align the semantic information with text modality. Finally, to further improve the quality of hash codes, we propose a discrete optimization algorithm to learn discrete hash codes directly, and each bit has a closed-form solution. Extensive experiments on multiple public databases show that our model can seamlessly incorporate attributes and achieve promising performance. [ABSTRACT FROM AUTHOR]

Recently, Shu et al. published “Numerical and Experimental Investigation of TID Radiation Effect on the Breakdown Voltage of 400 V SOI NLDMOS.” This short article is an extension of that earlier article. The breakdown voltage of 400-V silicon-on-insulator n-channel laterally diffused metal–oxide semiconductor field-effect transistors with different drift region lengths are examined after exposure to total ionizing dose. The variation observed is analyzed using technology computer-aided design simulation. [ABSTRACT FROM AUTHOR]

The hash join operator is one of the most important relational operations used in database. The offloading and acceleration of this operation on hardware has been a technique of growing interest for a long time. However, the non-uniform distribution of data caused by hash collisions negatively affects the throughput of the hash join algorithm, owing the variation in the number of hash table accesses required for each lookup. To resolve this issue, a non-collision parallel static random-access memory (SRAM)-based hash join architecture is presented. This architecture utilizes multiple hash functions and content addressable memories (CAMs) to eliminate hash collision, thereby ensuring a worst constant memory access for each phase in the hash join algorithm and consequently improving the hash join throughput. The proposed architecture was implemented on a Xilinx field programmable gate array (FPGA), and the experimental results show that our design achieved a high hash join throughput of 153.6 million tuples per second, and a speedup factor of at least 2.5 with the best existing FPGA-based hash join architecture and a match rate of 50%. [ABSTRACT FROM AUTHOR]

Various kinds of features prove to be effective for content-based image retrieval. However, due to the diversity of image contents, a descriptor may achieve impressive performance on specific images while becoming invalid on others. Although some efforts have been made to combine features as complementary counterparts, proper weighting scheme is still a challenge for fast and accurate retrieval. In this paper, we propose an effective fusion method, termed as Topo-correlation (Topo), where the importance of each feature is measured by cross-view correlations on local affinity graphs. Specifically, the weights of similarities are node-sensitive as well as modality-sensitive, thus boosting the results of good cues while depressing adverse factors for individual images. By estimating the consensus of similarity scores with regard to a query-driven criterion, the weighted graphs are generated efficiently with low computational complexity. Extensive experimental results on four benchmarks demonstrate the superiority of the proposed approach over the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

The number of superpixels (i.e., segmentation scale) is crucial for spectral–spatial hyperspectral image (HSI) classification. Existing methods always set the segmentation scale through a manually experimental strategy, which is time-consuming and unsuitable for various complicated practical applications. The information fusion of complementary multiple scales is proven to be more effective than the single scale for HSI classification, but the scale level is still set manually. In this article, we propose a novel adaptive multiscale segmentations (AMSs) method that can automatically provide a set of suitable scales that are adapted to different hyperspectral data. Specifically, based on the assumption that the segmentation scale of HSI is related to the image complexity itself, the texture ratio and the number of land cover classes are used to examine a candidate scale pool. A good scale means that it contains a small spectral difference between pixels within the same superpixel (intrasuperpixel discrimination index) and a large discrepancy between neighboring superpixels (intersuperpixel discrimination index). Thus, an intra–interscale discrimination index is defined and applied to depict the characteristics of the scale. Then, the scale with the best intra–inter discrimination index, which usually has satisfactory performance, is treated as the initially selected scale. The remaining suitable scales are iteratively compared with the selected ones and then added to the target scale pool, until the newly added scale can no longer provide significantly complementary information. Extensive experimental results on three HSI data sets have demonstrated the effectiveness of the proposed AMS when compared with state-of-the-art methods. [ABSTRACT FROM AUTHOR]

The breakdown voltage (BVDS) of uniform doping (UD) and linear variable doping (LVD) silicon-on-insulator (SOI) p-channel laterally diffused metal oxide semiconductor field effect transistors (p-LDMOSFETs) is examined after exposure to the total ionizing dose (TID). The results show BVDS degradation with accumulated dose. TID degradation with the MOS device biased off is more severe than that with the MOS device biased on. Also, LVD BVDS is more significant than UD BVDS at a given TID. Technology computer-aided design (TCAD) simulation is used to gain an insight into the mechanism causing BVDS degradation. [ABSTRACT FROM AUTHOR]

Recent years have witnessed the remarkable progress of applying deep learning models in video person re-identification (Re-ID). A key factor for video person Re-ID is to effectively construct discriminative and robust video feature representations for many complicated situations. Part-based approaches employ spatial and temporal attention to extract representative local features. While correlations between parts are ignored in the previous methods, to leverage the relations of different parts, we propose an innovative adaptive graph representation learning scheme for video person Re-ID, which enables the contextual interactions between relevant regional features. Specifically, we exploit the pose alignment connection and the feature affinity connection to construct an adaptive structure-aware adjacency graph, which models the intrinsic relations between graph nodes. We perform feature propagation on the adjacency graph to refine regional features iteratively, and the neighbor nodes’ information is taken into account for part feature representation. To learn compact and discriminative representations, we further propose a novel temporal resolution-aware regularization, which enforces the consistency among different temporal resolutions for the same identities. We conduct extensive evaluations on four benchmarks, i.e. iLIDS-VID, PRID2011, MARS, and DukeMTMC-VideoReID, experimental results achieve the competitive performance which demonstrates the effectiveness of our proposed method. Code is available at https://github.com/weleen/AGRL.pytorch. [ABSTRACT FROM AUTHOR]

Existing successful person re-identification (Re-ID) models often employ the part-level representation to extract the fine-grained information, but commonly use the loss that is particularly designed for global features, ignoring the relationship between semantic parts. In this paper, we present a novel triplet loss that emphasizes the salient parts and also takes the consideration of alignment. This loss is based on the crossing-bing matching metric that also known as Wasserstein Distance. It measures how much effort it would take to move the embeddings of local features to align two distributions, such that it is able to find an optimal transport matrix to re-weight the distance of different local parts. The distributions in support of local parts is produced via a new attention mechanism, which is calculated by the inner product between high-level global feature and local features, representing the importance of different semantic parts w.r.t. identification. We show that the obtained optimal transport matrix can not only distinguish the relevant and misleading parts, and hence assign different weights to them, but also rectify the original distance according to the learned distributions, resulting in an elegant solution for the mis-alignment issue. Besides, the proposed method is easily implemented in most Re-ID learning system with end-to-end training style, and can obviously improve their performance. Extensive experiments and comparisons with recent Re-ID methods manifest the competitive performance of our method. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel deep neural network based attention model to learn the representative local regions from a video sequence for person re-identification. Specifically, we propose a multi-scale spatial-temporal attention (MSTA) model to measure the regions of each frame in different scales from the perspective of whole video sequence. Compared to traditional temporal attention models, MSTA focuses on exploiting the importance of local regions of each frame to the whole video representation in both spatial and temporal domains. A new training strategy is designed for the proposed model by incorporating the image-to-image mode with the video-to-video mode. Extensive experiments on benchmark datasets demonstrate the superiority of the proposed model over state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Vehicle Re-Identification (Re-ID) is challenging because vehicles of the same model commonly show similar appearance. We tackle this challenge by proposing a Global-Regional Feature (GRF) that depicts extra local details to enhance discrimination power in addition to the global context. It is motivated by the observation that, vehicles of same color, maker, and model can be distinguished by their regional difference, e.g., the decorations on the windshields. To accelerate the GRF learning and promote its discrimination power, we propose a Group-Group Loss (GGL) to optimize the distance within and across vehicle image groups. Different from the siamese or triplet loss, GGL is directly computed on image groups rather than individual sample pairs or triplets. By avoiding traversing numerous sample combinations, GGL makes the model training easier and more efficient. Those two contributions highlight this work from previous methods on vehicle Re-ID task, which commonly learn global features with triplet loss or its variants. We evaluate our methods on two large-scale vehicle Re-ID datasets, i.e., VeRi and VehicleID. Experimental results show our methods achieve promising performance in comparison with recent works. [ABSTRACT FROM AUTHOR]

This article presents a tensor multi-task model for person re-identification (Re-ID). Due to discrepancy among cameras, our approach regards Re-ID from multiple cameras as different but related classification tasks, each task corresponding to a specific camera. In each task, we distinguish the person identity as a one-vs-all linear classification problem, where one classifier is associated with a specific person. By constructing all classifiers into a task-specific projection matrix, the proposed method could utilize all the matrices to form a tensor structure, and jointly train all the tasks in a uniform tensor space. In this space, by assuming the features of the same person under different cameras are generated from a latent subspace, and different identities under the same perspective share similar patterns, the high-order correlations, not only across different tasks but also within a certain task, can be captured by utilizing a new type of low-rank tensor constraint. Therefore, the learned classifiers transform the original feature vector into the latent space, where feature distributions across cameras can be well-aligned. Moreover, this model can be incorporated into multiple visual features to boost the performance, and easily extended to the unsupervised setting. Extensive experiments and comparisons with recent Re-ID methods manifest the competitive performance of our method. [ABSTRACT FROM AUTHOR]

Multi-view visual classification methods have been widely applied to use discriminative information of different views. This strategy has been proven very effective by many researchers. On the one hand, images are often treated independently without fully considering their visual and semantic correlations. On the other hand, view consistency is often ignored. To solve these problems, in this paper, we propose a novel multi-view image classification method with visual, semantic and view consistency (VSVC). For each image, we linearly combine multi-view information for image classification. The combination parameters are determined by considering both the classification loss and the visual, semantic and view consistency. Visual consistency is imposed by ensuring that visually similar images of the same view are predicted to have similar values. For semantic consistency, we impose the locality constraint that nearby images should be predicted to have the same class by multi-view combination. View consistency is also used to ensure that similar images have consistent multi-view combination parameters. An alternative optimization strategy is used to learn the combination parameters. To evaluate the effectiveness of VSVC, we perform image classification experiments on several public datasets. The experimental results on these datasets show the effectiveness of the proposed VSVC method. [ABSTRACT FROM AUTHOR]

Objects in unmanned aerial vehicle (UAV) images are generally small due to the high-photography altitude. Although many efforts have been made in object detection, how to accurately and quickly detect small objects is still one of the remaining open challenges. In this paper, we propose a feature fusion and scaling-based single shot detector (FS-SSD) for small object detection in the UAV images. The FS-SSD is an enhancement based on FSSD, a variety of the original single shot multibox detector (SSD). We add an extra scaling branch of the deconvolution module with an average pooling operation to form a feature pyramid. The original feature fusion branch is adjusted to be better suited to the small object detection task. The two feature pyramids generated by the deconvolution module and feature fusion module are utilized to make predictions together. In addition to the deep features learned by the FS-SSD, to further improve the detection accuracy, spatial context analysis is proposed to incorporate the object spatial relationships into object redetection. The interclass and intraclass distances between different object instances are computed as a spatial context, which proves effective for multiclass small object detection. Six experiments are conducted on the PASCAL VOC dataset and the two UAV image datasets. The experimental results demonstrate that the proposed method can achieve a comparable detection speed but an accuracy superior to those of the six state-of-the-art methods. [ABSTRACT FROM AUTHOR]

The electrical characteristics of silicon-on-insulator (SOI) n-channel laterally diffused metal oxide semiconductor field effect transistors (NLDMOSFETs) are examined after exposure to the total ionizing dose (TID). The devices are partially radiation hardened in that the gate oxide (GOX) and the field oxide (FOX) have undergone a hardening process, but not the buried oxide (BOX). Two bias conditions, OFF-state and all terminals grounded, during irradiation are examined. The irradiated device shows that the OFF-state leakage current (IL) of these devices increases with the accumulated dose. The physical mechanisms for the IL are analyzed by technology computer-aided design (TCAD) simulation. The OFF-state leakage current (IL) can be explained due to the effects of radiation-induced trapped charge at the BOX silicon interface. [ABSTRACT FROM AUTHOR]