Search

We consider the problem of predicting the impression that one subject has of another in a video clip showing a group of interacting people. Our novel Dyadic Impression Prediction System ( DIPS ) contains two major innovations. First, we develop a novel method to align the facial expressions of subjects p i and p j as well as account for the temporal delay that might be involved in p i reacting to p j ’s facial expressions. Second, we propose the concept of a multilayered stochastic network for impression prediction on top of which we build a novel Temporal Delayed Network graph neural network architecture. Our overall DIPS architecture predicts six dependent variables relating to the impression p i has of p j . Our experiments show that DIPS beats eight baselines from the literature, yielding statistically significant improvements of 19.9% to 30.8% in AUC and 12.6% to 47.2% in F1-score. We further conduct ablation studies showing that our novel features contribute to the overall quality of the predictions made by DIPS .

Identifying persuasive speakers in an adversarial environment is a critical task. In a national election, politicians would like to have persuasive speakers campaign on their behalf. When a company faces adverse publicity, they would like to engage persuasive advocates for their position in the presence of adversaries who are critical of them. Debates represent a common platform for these forms of adversarial persuasion. This paper solves two problems: the Debate Outcome Prediction (DOP) problem predicts who wins a debate while the Intensity of Persuasion Prediction (IPP) problem predicts the change in the number of votes before and after a speaker speaks. Though DOP has been previously studied, we are the first to study IPP. Past studies on DOP fail to leverage two important aspects of multimodal data: 1) multiple modalities are often semantically aligned, and 2) different modalities may provide diverse information for prediction. Our M2P2 (Multimodal Persuasion Prediction) framework is the first to use multimodal (acoustic, visual, language) data to solve the IPP problem. To leverage the alignment of different modalities while maintaining the diversity of the cues they provide, M2P2 devises a novel adaptive fusion learning framework which fuses embeddings obtained from two modules -- an alignment module that extracts shared information between modalities and a heterogeneity module that learns the weights of different modalities with guidance from three separately trained unimodal reference models. We test M2P2 on the popular IQ2US dataset designed for DOP. We also introduce a new dataset called QPS (from Qipashuo, a popular Chinese debate TV show ) for IPP. M2P2 significantly outperforms 4 recent baselines on both datasets., published in IEEE Trans. on Multimedia 2021

Tianyu Tan,1 Chao Zhou,1 Rongrong Lu,1 Cheng Chen,2 Chongyang Bai,1 Long Li,1 Guohui Wu1 1Chongqing Center for Disease Control and Prevention, Chongqing, 400042, People’s Republic of China; 2Chongqing Jiulongpo District Center for Disease Control and Prevention, Chongqing, 400039, People’s Republic of ChinaCorrespondence: Guohui Wu, Chongqing Center for Disease Control and Prevention, No. 8, Changjiang 2nd Road, Chongqing, 400042, People’s Republic of China, Tel +8602368801491, Email wgh68803652@163.comPurpose: This study aimed to assess the proportion of depression and its associated factors among men living with HIV/AIDS and aged 50 and over in Chongqing, China.Methods: This cross-sectional study included 774 men diagnosed as HIV/AIDS and aged 50 and over in four regions. Data were collected through face-to-face interviews with a structured questionnaire. Multivariate logistic regression analysis was performed to identify factors associated with depression. A two-tailed P-value less than 0.05 was considered as statistical significance.Results: A total of 293 participants (37.9%) reported depressive symptoms. Results from multivariate logistic regression suggested that individuals aged ≥ 70 (AOR = 1.99, 95% CI: 1.28– 3.08) and those living in rural areas (AOR = 2.79, 95% CI: 1.96– 3.97) were associated with higher odds of depression, while those being employed (AOR = 0.50, 95% CI: 0.32– 0.80) and with monthly income > 3000 CNY (AOR = 0.47, 95% CI: 0.30– 3.74) had lower odds of depression.Conclusion: Our findings indicate a high prevalence of depression among older men living with HIV/AIDS in Chongqing, China. Age, residence, employment, and income have been identified as potential risk factors of depressive symptoms. Mental-health intervention initiatives should be tailored to target adults with a higher risk of depression.Keywords: depression, China, older male, HIV/AIDS, influencing factors

The proliferation of visual information promoted in part by the Internet of Things is increasing demand for high-quality imaging, which in turn imposes more stringent physical requirements on photodetectors. Given that dark current is a significant figure of merit for photodetectors, we report herein a vertical ferroelectric semiconductor junction based on two-dimensional α-In2Se3 that suppresses the dark current and, thereby, enhances photodetection sensitivity. By utilizing the tight coupling between the ferroelectric and semiconductor properties of α-In2Se3, the two-terminal graphene-ferroelectric semiconductor–graphene crossbar structure demonstrates typical memristive behavior. The conductance, reflecting the dark current, is effectively regulated by modulating the height of the out-of-plane ferroelectric polarization-induced Schottky barrier height modulation between α-In2Se3 and graphene. As a result, the dark current is suppressed to 14 nA when α-In2Se3 is polarized down, which is a 50-fold decrease from 660 nA of dark current when α-In2Se3 is polarized up. Furthermore, α-In2Se3 exhibits excellent optoelectronic properties, demonstrating a high responsivity of 4.3 × 104 A/W, a fast response speed of 43 μs, and a broadband response spectrum from the visible to 980 nm. The combination of semiconductor and ferroelectric properties means that such devices may be used in self-powered, broadband, and highly integrated optoelectronic platforms.

Identifying the visual focus of attention (VFOA) in multi-person discussions is related to many types of social interactions such as dominance/deference, like/dislike, and trust/distrust relationships. However, identifying the VFOA of a person is challenging since it changes rapidly in dynamic discussions. We propose ICAF (Iterative Collective Attention Focus), a system that simultaneously tracks the VFOA and speaking probabilities of all people. In order to apply the system previously unseen videos, we propose a lightly supervised technique to train the model in ICAF with performance which is only slightly worse than in the fully supervised case. Our system can visualize the predicted VFOA and speaking probabilities and interaction networks in videos.

In this study, we address a cross-domain problem of applying computer vision approaches to reason about human facial behavior when people play The Resistance game. To capture the facial behaviors, we first collect several hours of video where the participants playing The Resistance game assume the roles of deceivers (spies) vs truth-tellers (villagers). We develop a novel attention-based neural network (NN) that advances the state of the art in understanding how a NN predicts the players’ roles. This is accomplished by discovering through learning those pixels and related frames which are discriminative and contributed the most to the NN’s inference. We demonstrate the effectiveness of our attention-based approach in discovering the frames and facial Action Units (AUs) that contributed to the NN’s class decision. Our results are consistent with the current communication theory on deception.

Using a novel dataset of Android banking trojans (ABTs), other Android malware, and goodware, we develop the $\sf {DBank}$ DBank system to predict whether a given Android APK is a banking trojan or not. We introduce the novel concept of a Triadic Suspicion Graph (TSG for short) which contains three kinds of nodes: goodware, banking trojans, and API packages. We develop a novel feature space based on two classes of scores derived from TSGs: suspicion scores (SUS) and suspicion ranks (SR)—the latter yields a family of features that generalize PageRank. While TSG features (based on SUS/SR scores) provide very high predictive accuracy on their own in predicting recent (2016-2017) ABTs, we show that the combination of TSG features with previously studied lightweight static and dynamic features in the literature yields the highest accuracy in distinguishing ABTs from goodware, while preserving the same accuracy of prior feature combinations in distinguishing ABTs from other Android malware. In particular, $\sf {DBank}$ DBank ’s overall accuracy in predicting whether an APK is a banking trojan or not is up to 99.9% AUC with 0.3% false positive rate. Moreover, we have already reported two unlabeled APKs from VirusTotal (which $\sf {DBank}$ DBank has detected as ABTs) to the Google Android Security Team—in one case, we discovered it before any of the 63 anti-virus products on VirusTotal did, and in the other case, we beat 62 of 63 anti-viruses on VirusTotal. This suggests that $\sf {DBank}$ DBank is capable of making new discoveries in the wild before other established vendors. We also show that our novel TSG features have some interesting defensive properties as they are robust to knowledge of the training set by an adversary: even if the adversary uses 90% of our training set and uses the exact TSG features that we use, it is difficult for him to infer $\sf {DBank}$ DBank ’s predictions on APKs. We additionally identify the features that best separate and characterize ABTs from goodware as well as from other Android malware. Finally, we develop a detailed data-driven analysis of five major recent ABT families: FakeToken , Svpeng , Asacub , BankBot , and Marcher , and identify the features that best separate them from goodware and other-malware.

To improve the accessibility of smart devices and to simplify their usage, building models which understand user interfaces (UIs) and assist users to complete their tasks is critical. However, unique challenges are proposed by UI-specific characteristics, such as how to effectively leverage multimodal UI features that involve image, text, and structural metadata and how to achieve good performance when high-quality labeled data is unavailable. To address such challenges we introduce UIBert, a transformer-based joint image-text model trained through novel pre-training tasks on large-scale unlabeled UI data to learn generic feature representations for a UI and its components. Our key intuition is that the heterogeneous features in a UI are self-aligned, i.e., the image and text features of UI components, are predictive of each other. We propose five pretraining tasks utilizing this self-alignment among different features of a UI component and across various components in the same UI. We evaluate our method on nine real-world downstream UI tasks where UIBert outperforms strong multimodal baselines by up to 9.26% accuracy., Comment: 8 pages, IJCAI 2021

Visual focus of attention in multi-person discussions is a crucial nonverbal indicator in tasks such as inter-personal relation inference, speech transcription, and deception detection. However, predicting the focus of attention remains a challenge because the focus changes rapidly, the discussions are highly dynamic, and the people's behaviors are inter-dependent. Here we propose ICAF (Iterative Collective Attention Focus), a collective classification model to jointly learn the visual focus of attention of all people. Every person is modeled using a separate classifier. ICAF models the people collectively---the predictions of all other people's classifiers are used as inputs to each person's classifier. This explicitly incorporates inter-dependencies between all people's behaviors. We evaluate ICAF on a novel dataset of 5 videos (35 people, 109 minutes, 7604 labels in all) of the popular Resistance game and a widely-studied meeting dataset with supervised prediction. See our demo at https://cs.dartmouth.edu/dsail/demos/icaf. ICAF outperforms the strongest baseline by 1%--5% accuracy in predicting the people's visual focus of attention. Further, we propose a lightly supervised technique to train models in the absence of training labels. We show that light-supervised ICAF performs similar to the supervised ICAF, thus showing its effectiveness and generality to previously unseen videos.

We consider the problems of predicting (i) the most dominant person in a group of people, and (ii) the more dominant of a pair of people, from videos depicting group interactions. We introduce a novel family of variables called Dominance Rank. We combine features not previously used for dominance prediction (e.g., facial action units, emotions), with a novel ensemble-based approach to solve these two problems. We test our models against four competing algorithms in the literature on two datasets and show that our results improve past performance. We show 2.4% to 16.7% improvement in AUC compared to baselines on one dataset, and a gain of 0.6% to 8.8% in accuracy on the other. Ablation testing shows that Dominance Rank features play a key role.

Most work on automated deception detection (ADD) in video has two restrictions: (i) it focuses on a video of one person, and (ii) it focuses on a single act of deception in a one or two minute video. In this paper, we propose a new ADD framework which captures long term deception in a group setting. We study deception in the well-known Resistance game (like Mafia and Werewolf) which consists of 5-8 players of whom 2-3 are spies. Spies are deceptive throughout the game (typically 30-65 minutes) to keep their identity hidden. We develop an ensemble predictive model to identify spies in Resistance videos. We show that features from low-level and high-level video analysis are insufficient, but when combined with a new class of features that we call LiarRank, produce the best results. We achieve AUCs of over 0.70 in a fully automated setting. Our demo can be found at http://home.cs.dartmouth.edu/~mbolonkin/scan/demo/, Comment: ICME 2019

PolyCubes provide compact representations for closed complex shapes and are essential to many computer graphics applications. Existing automatic PolyCube construction methods usually suffer from poor quality or time-consuming computation. In this paper, we provide a highly efficient method to compute volumetric PolyCube-maps. Given an input tetrahedral mesh, we utilize two novel normal-driven volumetric deformation schemes and a polycube-allowable mesh segmentation to drive the input to a volumetric PolyCube structure. Our method can robustly generate foldover-free and low-distortion PolyCube-maps in practice, and provide a flexible control on the number of corners of Polycubes. Compared with state-of-the-art methods, our method is at least one order of magnitude faster and has better mapping qualities. We demonstrate the efficiency and efficacy of our method in PolyCube construction and all-hexahedral meshing on various complex models.

Aircraft Scheduling in terminal areas is the key technology for diminishing delay and cost. Especially in SATS (Small Aircraft Transportation System), without the guidance of ATC (Air Traffic Control), efficiency is hard to reach. Considering diversity of aircrafts in SATS, traditional approaches to solve these problems, such as CPS (Constraint Position Switch) is not available. Promote to sequence aircrafts based on flying ability, and improve genetic algorithm by introduce local fitness value, using it as optimize function. Also, take vortex and conflict-free as the constraint. A potential algorithm is introduced to de-conflict for its ability to cover all possible conflict scenarios involving multiple agents. As a dynamic system, the cost of resolve conflict during the approach will feed back to the system to rescheduling. Simulation shows, a conflict-free sequence with lower delay is reached. Besides, the improved algorithm is more directive and convergence quickly, accelerating solving process. It can meet the application's needs in real-time.