Search

In steady-state visual-evoked potential (SSVEP) based brain-computer interfaces (BCIs), existing detection algorithms utilizing spatial filters like task-related component analysis (TRCA) derive the spatial filters mainly through maximizing the inter-trial similarity between the combined signals over the training set. Although they achieve by far the best classification performance in SSVEP-based BCIs, some important problems are still unresolved. Especially, the mechanism of how spatial filters cancel the background noise in brain signals and optimize the signal-to-noise ratio (SNR) of SSVEPs is still not figured out. Therefore, to solve these problems, in this paper a new perspective of spatial filter design is proposed. Specifically, a linear generative signal model of SSVEP is adopted and the spatial filters are obtained automatically through maximum likelihood estimation of source signals and channel vectors. In the same time, the relation between maximum likelihood estimation and signal-to-noise ratio (SNR) maximization is discussed. Through a step-by-step formulation, this paper provides a theoretical justification for those conventional algorithms utilizing spatial filters. As for the classification performance, the proposed scheme is tested on a benchmark dataset of 35 subjects. Experiment results show that the classification performance of the proposed scheme is competitive against three benchmark algorithms, which include TRCA. Especially, the proposed scheme achieves a fair performance improvement over the benchmark methods in the cases where a shorter time window, or a larger number of electrodes, or a smaller number of training blocks are adopted. [ABSTRACT FROM AUTHOR]

Inspired by the great success of machine learning (ML), researchers have applied ML techniques to visualizations to achieve a better design, development, and evaluation of visualizations. This branch of studies, known as ML4VIS, is gaining increasing research attention in recent years. To successfully adapt ML techniques for visualizations, a structured understanding of the integration of ML4VIS is needed. In this article, we systematically survey 88 ML4VIS studies, aiming to answer two motivating questions: “what visualization processes can be assisted by ML?” and “how ML techniques can be used to solve visualization problems? ”This survey reveals seven main processes where the employment of ML techniques can benefit visualizations: Data Processing4VIS, Data-VIS Mapping, Insight Communication, Style Imitation, VIS Interaction, VIS Reading, and User Profiling. The seven processes are related to existing visualization theoretical models in an ML4VIS pipeline, aiming to illuminate the role of ML-assisted visualization in general visualizations. Meanwhile, the seven processes are mapped into main learning tasks in ML to align the capabilities of ML with the needs in visualization. Current practices and future opportunities of ML4VIS are discussed in the context of the ML4VIS pipeline and the ML-VIS mapping. While more studies are still needed in the area of ML4VIS, we hope this article can provide a stepping-stone for future exploration. A web-based interactive browser of this survey is available at https://ml4vis.github.io. [ABSTRACT FROM AUTHOR]

Given a natural language expression and an image/video, the goal of referring segmentation is to produce the pixel-level masks of the entities described by the subject of the expression. Previous approaches tackle this problem by implicit feature interaction and fusion between visual and linguistic modalities in a one-stage manner. However, human tends to solve the referring problem in a progressive manner based on informative words in the expression, i.e., first roughly locating candidate entities and then distinguishing the target one. In this paper, we propose a cross-modal progressive comprehension (CMPC) scheme to effectively mimic human behaviors and implement it as a CMPC-I (Image) module and a CMPC-V (Video) module to improve referring image and video segmentation models. For image data, our CMPC-I module first employs entity and attribute words to perceive all the related entities that might be considered by the expression. Then, the relational words are adopted to highlight the target entity as well as suppress other irrelevant ones by spatial graph reasoning. For video data, our CMPC-V module further exploits action words based on CMPC-I to highlight the correct entity matched with the action cues by temporal graph reasoning. In addition to the CMPC, we also introduce a simple yet effective Text-Guided Feature Exchange (TGFE) module to integrate the reasoned multimodal features corresponding to different levels in the visual backbone under the guidance of textual information. In this way, multi-level features can communicate with each other and be mutually refined based on the textual context. Combining CMPC-I or CMPC-V with TGFE can form our image or video version referring segmentation frameworks and our frameworks achieve new state-of-the-art performances on four referring image segmentation benchmarks and three referring video segmentation benchmarks respectively. Our code is available at https://github.com/spyflying/CMPC-Refseg. [ABSTRACT FROM AUTHOR]

This paper studies the problem of learning the conditional distribution of a high-dimensional output given an input, where the output and input may belong to two different domains, e.g., the output is a photo image and the input is a sketch image. We solve this problem by cooperative training of a fast thinking initializer and slow thinking solver. The initializer generates the output directly by a non-linear transformation of the input as well as a noise vector that accounts for latent variability in the output. The slow thinking solver learns an objective function in the form of a conditional energy function, so that the output can be generated by optimizing the objective function, or more rigorously by sampling from the conditional energy-based model. We propose to learn the two models jointly, where the fast thinking initializer serves to initialize the sampling of the slow thinking solver, and the solver refines the initial output by an iterative algorithm. The solver learns from the difference between the refined output and the observed output, while the initializer learns from how the solver refines its initial output. We demonstrate the effectiveness of the proposed method on various conditional learning tasks, e.g., class-to-image generation, image-to-image translation, and image recovery. The advantage of our method over GAN-based methods is that our method is equipped with a slow thinking process that refines the solution guided by a learned objective function. [ABSTRACT FROM AUTHOR]

Full projector compensation aims to modify a projector input image to compensate for both geometric and photometric disturbance of the projection surface. Traditional methods usually solve the two parts separately and may suffer from suboptimal solutions. In this paper, we propose the first end-to-end differentiable solution, named CompenNeSt++, to solve the two problems jointly. First, we propose a novel geometric correction subnet, named WarpingNet, which is designed with a cascaded coarse-to-fine structure to learn the sampling grid directly from sampling images. Second, we propose a novel photometric compensation subnet, named CompenNeSt, which is designed with a siamese architecture to capture the photometric interactions between the projection surface and the projected images, and to use such information to compensate the geometrically corrected images. By concatenating WarpingNet with CompenNeSt, CompenNeSt++ accomplishes full projector compensation and is end-to-end trainable. Third, to improve practicability, we propose a novel synthetic data-based pre-training strategy to significantly reduce the number of training images and training time. Moreover, we construct the first setup-independent full compensation benchmark to facilitate future studies. In thorough experiments, our method shows clear advantages over prior art with promising compensation quality and meanwhile being practically convenient. [ABSTRACT FROM AUTHOR]

Instance segmentation needs to locate all instances in an image correctly and segment each instance precisely. Currently, the most dominant methods for instance segmentation take object detection as a pre-task. However, they rely on the accuracy of object detection incredibly. If the pre-task cannot predict an accurate bounding box, the performance of instance segmentation will degenerate. In this paper, we present a novel method for instance segmentation to solve this problem, which is called Segmenting Beyond the Bounding Box (S3B-Net). Our S3B-Net designs a sub-network to help instance segmentation methods based on object detection to segment the part of an instance beyond the bounding box. Specifically, the sub-network first predicts a two-dimensional pixel embedding for each pixel. Then, the Gaussian function is employed to calculate a pixel’s probability belongs to a corresponding instance according to the two-dimensional pixel embedding. Finally, the output of the sub-network combines with the output of instance segmentation based on object detection to generate a more precise instance mask. Our sub-network can easily extend on the existing instance segmentation method based on object detection to segment instance beyond the bounding box. We do our experiments on dominant instance segmentation datasets, such as the COCO dataset and Cityscapes dataset. The results show that our method can achieve 6.8 points gain compared with the baseline Mask R-CNN with ResNet-50-FPN in Cityscapes datasets, and 1.7 points gain with ResNet-101-FPN-DCN in COCO datasets. Our S3B-Net outperforms the previous state-of-the-art instance segmentation method, which proves our method is competitive. The source code of our method will be made available. [ABSTRACT FROM AUTHOR]

With the increase of mobile devices, Mobile Crowdsensing (MCS) has become an efficient way to ubiquitously sense and collect environment data. Comparing to traditional sensor networks, MCS has a vital advantage that workers play an active role in collecting and sensing data. However, due to the openness of MCS, workers and sensors are of different qualities. Low quality sensors and workers may yield noisy data or even inaccurate data. Which gives the importance of inferring the quality of workers and sensors and seeking a valid task assignment with enough total qualities for MCS. To solve the problem, we adopt truth inference methods to iteratively infer the truth and qualities. Based on the quality inference, this paper proposes a task assignment problem called quality-bounded task assignment with redundancy constraint (QTAR). Different from traditional task assignment problem, redundancy constraint is added to satisfy the preliminaries of truth inference, which requires that each task should be assigned a certain or more amount of workers. We prove that QTAR is NP-complete and propose a $ (2+\epsilon)$ (2 + ε) - approximation algorithm for QTAR, called QTA. Finally, experiments are conducted on both synthesis data and real dataset. The results of the experiments prove the efficiency and effectiveness of our algorithms. [ABSTRACT FROM AUTHOR]

Survival analysis aims at predicting the time to event of interest along with its probability on longitudinal data. It is commonly used to make predictions for a single specific event of interest at a given time point. However, predicting the occurrence of multiple events of interest simultaneously and dynamically is needed in many real-world applications. An intuitive way to solve this problem is to simply apply the standard survival analysis method independently to each prediction task at each time point. However, it often leads to a sub-optimal solution since the underlying dependencies between these tasks are ignored. This motivates us to analyze these prediction tasks jointly in order to select the common features shared across all the tasks. In this paper, we formulate a temporal (Multiple Time points) Multi-Task learning framework (MTMT) for survival analysis problems using tensor representation. More specifically, given a survival dataset and a sequence of time points, which are considered as the monitored time points for the events of interest, we reformulate the survival analysis problem to jointly handle each task at each time point and optimize them simultaneously. We demonstrate the performance of the proposed MTMT model on important real-world datasets, including employee attrition and medical records. We show the superior performance of the MTMT model compared to several state-of-the-art models using standard metrics. We also provide the list of important features selected by our MTMT model thus demonstrating the interpretability of the proposed model. [ABSTRACT FROM AUTHOR]

To meet the execution requirements of delay-sensitive services in vehicular edge computing (VEC) networks, vehicular services need to be offloaded to edge computing nodes. For complex, large-scale services, the services need to be migrated if the services are not completed before the vehicles leave the coverage of edge computing nodes. Trust and resource matching between areas thus become major problems. This paper studies the decision model of vehicular service offloading and migration. First, software-defined network (SDN) technology is introduced into the traditional network architecture, and a two-layer distributed SDN-controlled VEC network architecture is designed, which is divided into a domain control layer and an area control layer. In this framework, we use the consortium blockchain as a carrier to share network topology information between SDN controllers to prevent information leakage. We then established a service offloading and migration optimization problem model to minimize service execution delay, reduce energy consumption and maximize the throughput of the blockchain system. We describe the problem model as a Markov Decision Process (MDP), introduce a deep reinforcement learning (DRL) algorithm named asynchronous advantage actor-critic (A3C) and design a dynamic service offloading and migration algorithm (DSOMA) based on A3C to solve the problem. Simulation results show that DSOMA can increase the throughput of the blockchain system, and DSOMA is superior to the deep Q-learning (DQN) algorithm and greedy offloading algorithm in reducing service execution delay and system energy consumption. [ABSTRACT FROM AUTHOR]

The existing state-of-the-art (SOTA) video salient object detection (VSOD) models have widely followed short-term methodology, which dynamically determines the balance between spatial and temporal saliency fusion by solely considering the current consecutive limited frames. However, the short-term methodology has one critical limitation, which conflicts with the real mechanism of our visual system — a typical long-term methodology. As a result, failure cases keep showing up in the results of the current SOTA models, and the short-term methodology becomes the major technical bottleneck. To solve this problem, this paper proposes a novel VSOD approach, which performs VSOD in a complete long-term way. Our approach converts the sequential VSOD, a sequential task, to a data mining problem, i.e., decomposing the input video sequence to object proposals in advance and then mining salient object proposals as much as possible in an easy-to-hard way. Since all object proposals are simultaneously available, the proposed approach is a complete long-term approach, which can alleviate some difficulties rooted in conventional short-term approaches. In addition, we devised an online updating scheme that can grasp the most representative and trustworthy pattern profile of the salient objects, outputting framewise saliency maps with rich details and smoothing both spatially and temporally. The proposed approach outperforms almost all SOTA models on five widely used benchmark datasets. [ABSTRACT FROM AUTHOR]

The stable continuation of trajectories among different targets has always been the key to the tracking performance of multi-object tracking (MOT) tasks. If features of the target are aggregated and classified simply, the discriminant features of the target will be ignored. This will affect the robustness of the trajectory generated by the model. Meanwhile, many popular models are keen to execute detection and feature extraction tasks in parallel. But these two tasks will conflict with each other when optimized respectively. Therefore, we propose our tracker MOTFR to solve the above problems. In this paper, we propose a Locally Shared Information Decoupling Module (LSIDM) to reduce task optimization conflicts while ensuring the necessary information sharing. Meanwhile, a feature recoding module for deep extraction of identity discriminative features is proposed, which is called the Feature Purification Module (FPM). By combining LSIDM and FPM modules, the model utilizes the discriminative appearance features to guide the optimization of detection and further improves the performance of our model. To solve the problem of targets disappearing due to various abnormal occlusion, a Short-term Trajectory Online Complement Strategy (STOCS) is proposed to realize the trajectories continuation of these targets in the tracking stage. Through sufficient experiments, we demonstrate the superior performance of our MOTFR, which guarantees high-quality detection while achieving the stability of the target trajectory. [ABSTRACT FROM AUTHOR]

Differential privacy (DP) has gained popularity in truth discovery recently due to its strong privacy guarantee. However, existing DP mechanisms for streaming data publication are not suitable for truth discovery as they fail to consider the different reliabilities of individuals, while the DP-based approaches for truth discovery are not suitable for streaming data because they ignore the correlations between truths over time. Directly applying these existing methods to streaming crowdsourced data would lead to low accuracy of the discovered truth. To solve this problem, in this paper, we propose an edge computing based privacy-preserving truth discovery mechanism, named PrivSTD, for streaming crowdsourced data to realize high accuracy of discovered truth while protecting the privacy of workers. Specifically, edge servers are introduced between the untrusted cloud server and workers to securely calculate the local truths and workers’ reliabilities. A truth-dependent budget recycle mechanism is proposed for each edge server to adaptively determine the perturbed timestamp and allocate the privacy budget according to the changing pattern of local truths. Besides, a reliability-based perturbation mechanism is proposed to reduce the perturbation magnitude on the basis of worker's reliability. We theoretical analyze the data utility and computation cost of PrivSTD, and prove that PrivSTD can satisfy $w$ w -event ($\epsilon,\delta$ ε , δ )-differential privacy. Extensive experimental results on synthetic and real-world datasets demonstrate that PrivSTD achieves better utility than the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

This paper proposes a gaze correction and animation method for high-resolution, unconstrained portrait images, which can be trained without the gaze angle and the head pose annotations. Common gaze-correction methods usually require annotating training data with precise gaze, and head pose information. Solving this problem using an unsupervised method remains an open problem, especially for high-resolution face images in the wild, which are not easy to annotate with gaze and head pose labels. To address this issue, we first create two new portrait datasets: CelebGaze ($256 \times 256$) and high-resolution CelebHQGaze ($512 \times 512$). Second, we formulate the gaze correction task as an image inpainting problem, addressed using a Gaze Correction Module (GCM) and a Gaze Animation Module (GAM). Moreover, we propose an unsupervised training strategy, i.e., Synthesis-As-Training, to learn the correlation between the eye region features and the gaze angle. As a result, we can use the learned latent space for gaze animation with semantic interpolation in this space. Moreover, to alleviate both the memory and the computational costs in the training and the inference stage, we propose a Coarse-to-Fine Module (CFM) integrated with GCM and GAM. Extensive experiments validate the effectiveness of our method for both the gaze correction and the gaze animation tasks in both low and high-resolution face datasets in the wild and demonstrate the superiority of our method with respect to the state of the art. [ABSTRACT FROM AUTHOR]

Recent years have witnessed a big leap in automatic visual saliency detection attributed to advances in deep learning, especially Convolutional Neural Networks (CNNs). However, inferring the saliency of each image part separately, as was adopted by most CNNs methods, inevitably leads to an incomplete segmentation of the salient object. In this paper, we describe how to use the property of part-object relations endowed by the Capsule Network (CapsNet) to solve the problems that fundamentally hinge on relational inference for visual saliency detection. Concretely, we put in place a two-stream strategy, termed Two-Stream Part-Object RelaTional Network (TSPORTNet), to implement CapsNet, aiming to reduce both the network complexity and the possible redundancy during capsule routing. Additionally, taking into account the correlations of capsule types from the preceding training images, a correlation-aware capsule routing algorithm is developed for more accurate capsule assignments at the training stage, which also speeds up the training dramatically. By exploring part-object relationships, TSPORTNet produces a capsule wholeness map, which in turn aids multi-level features in generating the final saliency map. Experimental results on five widely-used benchmarks show that our framework consistently achieves state-of-the-art performance. The code can be found on https://github.com/liuyi1989/TSPORTNet. [ABSTRACT FROM AUTHOR]

Near-infrared and visual (NIR-VIS) face matching, as the most typical task in Heterogeneous Face Recognition (HFR), has attracted increasing attention in recent years. However, due to the large within-class discrepancies, including domain differences and residual discrepancies (i.e., lighting, expressions, occlusion, blurry, pose, etc), this is still a difficult task. Conventional NIR-VIS FR methods only focus on reducing the modality gap between cross-domain images, while neglecting to eliminate the residual variations. To better solve the above problems, this paper proposes a novel Orthogonal Modality Disentanglement and Representation Alignment (OMDRA) approach, which consists of three key components, including Modality-Invariant (MI) loss, Orthogonal Modality Disentanglement (OMD) and Deep Representation Alignment (DRA). Firstly, the MI loss is designed to learn modality-invariant and identity-discriminative representation, by increasing between-class separability and within-class compactness between NIR and VIS heterogeneous data. Secondly, the high-level Hybrid Facial Feature (HFF) layer of the backbone network is projected into two subspaces: the modality-related and identity-related subspaces. The OMD is designed to decouple modal information via an adversarial process, and we further impose Orthogonal Representation Decorrelation (ORD) to the OMD to decrease the correlation between identity representations and domain representations, as well as enhancing their representation capabilities. Finally, the DRA aims to eliminate the residual variations by performing a high-level representation alignment between non-neutral face and neutral face, which can effectively guides the network to learn discriminative and residual-invariant face representation. The joint scheme enables the disentanglement of modality variations, elimination of residual discrepancies, and the purification of identity information. Extensive experiments on challenging cross-domain databases indicate that our OMDRA method is superior to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Mobile-edge computing (MEC) has been recognized as a promising solution to provide efficient communication and computation capabilities for mobile users (MUs). However, the problem relating to parallel offloading and load balancing with multiple cooperative MEC servers and massive delay-sensitive execution workloads remains to be investigated. In this paper, we study a joint parallel offloading and load balancing policy for such an MEC system. We formulate a long-term system average-cost (i.e., weighted sum of energy consumption and execution delays) stochastic programming problem under MU battery level stability and delay constraints. Our aims include optimizing the data sizes for uplink offloading transmissions for optimized communication and computation resource allocation among multi-users, and maximizing the computation capability utilization of cooperative MEC servers by load balancing. To solve this problem, we first design a Lyapunov-based centralized cost management algorithm (LYP-CCMA) to obtain the optimal system average-cost under the battery level stability constraints. Further, we propose two algorithms based on alternating direction method of multipliers (ADMM) to implement distributed resources allocation. Simulation results verify our analysis and demonstrate the superior performance of our proposed schemes over several baseline schemes. [ABSTRACT FROM AUTHOR]

In this paper, the legitimate surveillance of a suspicious mobile edge computing (MEC) network consisting of a suspicious edge server (SES) and multiple suspicious users (SUs), in the presence of a full-duplex monitor is studied. Each SU has a computation task to complete within a time deadline and can completely or partially offload the task to SES, while the monitor can either jam or assist the suspicious communications during task uploading and result downloading. With the heterogeneous offloading model adopted by the SUs, the problem of optimizing the monitor mode and transmit power to maximize the average ratio of successfully eavesdropped tasks, subject to the monitor transmit power constraint and the task completion time deadline constraint is investigated. The problem is solved via exploring the particular problem structure and adopting the sum-of-ratios optimization. Simulation results show that the proposed algorithm significantly outperforms the benchmark algorithms, especially for the SUs with partial offloading. It is also shown that the proposed algorithm is of low-complexity and achieves almost the same performance as the high-complexity optimal algorithm. Besides, compared to the SUs with binary offloading, the eavesdropping performance for the SUs with partial offloading is shown to be much better. [ABSTRACT FROM AUTHOR]

Existing Space-Terrestrial Integrated Network (STIN) applications collect all data from multiple satellites and terrestrial nodes to the specific analyze center on the earth for processing, which wastes lots of network resources. To save these resources, we propose a novel processing-while-transmitting pattern in the SDN-based STIN architecture. Through a logically centralized control plane, it cooperatively processes a complex task on appropriate nodes during data transmission. Here, the key point is to jointly determine the transmission path and place subtasks adaptive to data distributions, heterogeneous link costs, task characteristics, the dynamic topology, and network resources. In this paper, we firstly formulate the Transmission-cost-minimized joint Routing and Tasks placement Problem (TRTP) in time-varying STINs. We prove it is NP-hard and has no Polynomial-Time Approximation Scheme (PTAS). To solve the problem, we propose the Joint Routing and Task Placement (JRTP) algorithm. It first converts the time-varying STIN to a stable graph to cope with the network dynamics, according to the topology and resources during task processing. Then, it jointly decides the routing and task placement through a task-topology graph model, which converts the TRTP problem on the stable graph to the classic shortest path problem. We prove that the performance of JRTP is bounded in cases when transmission resources are sufficient and further improve it through the idea of reinforcement. The experimental results show that our processing pattern can significantly decrease the transmission cost and delay, and our algorithms outperform most related ones. [ABSTRACT FROM AUTHOR]

Zero-shot classification is a promising paradigm to solve an applicable problem when the training classes and test classes are disjoint. Achieving this usually needs experts to externalize their domain knowledge by manually specifying a class-attribute matrix to define which classes have which attributes. Designing a suitable class-attribute matrix is the key to the subsequent procedure, but this design process is tedious and trial-and-error with no guidance. This paper proposes a visual explainable active learning approach with its design and implementation called semantic navigator to solve the above problems. This approach promotes human-AI teaming with four actions (ask, explain, recommend, respond) in each interaction loop. The machine asks contrastive questions to guide humans in the thinking process of attributes. A novel visualization called semantic map explains the current status of the machine. Therefore analysts can better understand why the machine misclassifies objects. Moreover, the machine recommends the labels of classes for each attribute to ease the labeling burden. Finally, humans can steer the model by modifying the labels interactively, and the machine adjusts its recommendations. The visual explainable active learning approach improves humans' efficiency of building zero-shot classification models interactively, compared with the method without guidance. We justify our results with user studies using the standard benchmarks for zero-shot classification. [ABSTRACT FROM AUTHOR]

Emerging crowdsensing paradigm enables a large number of sensing applications, where much attention is drawn to the fundamental problems of data collection and truth inference. Existing works have devised manifold techniques to discover truth from collected noisy data, but they frequently ignore various expertise of workers and dynamic information of crowdsensing system, thus leading to error-prone estimated truth and unqualified sensing data. In this paper, we design an online location-aware crowdsensing system to accurately estimate truth and efficiently assign tasks. Specifically, we unify diverse types of numerical and categorical tasks based on probabilistic graphical model, and then propose unsupervised learning methods which can dynamically infer ground truth and various worker expertise at the same time. Furthermore, we develop online task allocation schemes to gradually gather high quality data considering location awareness and inferred worker expertise. In particular, we convert the complicated task allocation into the additive form of probability improvement and entropy reduction, thereby solving the allocation problem via linearly selecting worker-task pair with low computation complexity. We finally carry out extensive evaluations using two datasets collected by our smartphones, where results demonstrate the superiority of our algorithms over the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

The sixth-generation (6G) of wireless communications systems will significantly rely on fog/edge network architectures for service provisioning. To realize this vision, AI-based fog/edge enabled reinforcement solutions are needed to serve highly stringent applications using dynamically varying resources. In this paper, we propose a cognitive dynamic fog/edge network where primary nodes (PNs) temporarily share their resources and act as fog nodes (FNs) for secondary nodes (SNs). Under this architecture, that unleashes multiple access opportunities, we design distributed fog probing schemes for SNs to search for available connections to access neighbouring FNs. Since the availability of these connections varies in time, we develop strategies to enhance the robustness to the uncertain availability of channels and fog nodes, and reinforce the connections with the FNs. A robustness control optimization is formulated with the aim to maximize the expected total long-term reliability of SNs’ transmissions. The problem is solved by an online robustness control (ORC) algorithm that involves online fog probing and an index-based connectivity activation policy derived from restless multi-armed bandits (RMABs) model. Simulation results show that our ORC scheme significantly improves the network robustness, the connectivity reliability and the number of completed transmissions. In addition, by activating the connections with higher indexes, the total long-term reliability optimization problem is solved with low complexity. [ABSTRACT FROM AUTHOR]

The minimum travel distance of task participants is one of the significant optimization objectives of privacy-preserving task assignment in mobile crowdsensing (MCS). However, when the travel distance is minimized, most of the previous schemes only focus on the task participant privacy and disregard the task requester privacy. Moreover, existing solutions usually only support the constraint of a single type, such as equality constraints or range constraints. In this paper, we propose a bilateral privacy-preserving Task Assignment mechanism for MCS (iTAM), which protects not only the task participants privacy but also the task requesters privacy and can minimize the travel distance. Furthermore, iTAM provides both equality and range constraints of task assignment by utilizing the Paillier cryptosystem. To accommodate the multiple relations between the task participants and the task, we propose the single/multiple task participants selection problems for a task requiring task participants to compete and cooperate. Experimental evaluations over synthetic and real-world data illustrate that iTAM is feasible and effective. Compared with the state-of-the-art, iTAM positively solves the optimal problem of travel distance. The complexities of iTAM are $\mathcal {O}(n)$ O (n) and $\mathcal {O}(n\log n)$ O (n log n) for a single and multiple task participants selection problems, respectively. [ABSTRACT FROM AUTHOR]

Unmanned aerial vehicle (UAV)-enabled mobile edge computing has been recognized as a promising technology to flexibly and efficiently handle computation-intensive and latency-sensitive tasks in the era of fifth generation (5G) and beyond. In this paper, we study the problem of Service Provisioning for UAV-enabled mobile edge computiNg (SPUN). Specifically, under task latency requirements and various resource constraints, we jointly optimize the service placement, UAV movement trajectory, task scheduling, and computation resource allocation, to minimize the overall energy consumption of all terrestrial user equipments (UEs). Due to the non-convexity of the SPUN problem as well as complex coupling among mixed integer variables, it is a non-convex mixed integer nonlinear programming (MINLP) problem. To solve this challenging problem, we propose two alternating optimization-based suboptimal solutions with different time complexities. In the first solution with relatively high complexity in the worst case, the joint service placement and task scheduling subproblem, and UAV trajectory subproblem are iteratively solved by the Branch and Bound (BnB) method and successive convex approximation (SCA), respectively, while the optimal solution to the computation resource allocation subproblem is efficiently obtained in the closed form. To avoid the high complexity caused by BnB, in the second solution, we propose a novel approximation algorithm based on relaxation and randomized rounding techniques for the joint service placement and task scheduling subproblem, while the other two subproblems are solved in the same way as that of the first solution. Extensive simulations demonstrate that the proposed solutions achieve significantly lower energy consumption of UEs compared to three benchmarks. [ABSTRACT FROM AUTHOR]

Unmanned aerial vehicle (UAV) communications and networks are of utmost concern. However, they have challenges such as the limited on-board energy and restricted transmit power. In this paper, we study a UAV-enabled communication scenario that a set of UAVs perform a virtual antenna array (VAA) to communicate with different remote base stations (BSs) by using collaborative beamforming (CB). To achieve a better transmission performance, the UAV elements can fly to optimal positions by using optimal speeds and adjust to optimal excitation current weights for performing CB transmissions. However, there are some trade-offs between energy consumption and transmission performance. Thus, we formulate a time and energy minimization communication multi-objective optimization problem (TEMCMOP) of CB in UAV networks to simultaneously minimize the total transmission time, total performing time of VAAs and total motion and hovering energy consumptions of UAVs by jointly optimizing the positions, flight speeds and excitation current weights of UAVs, as well as the order of communicating with different BSs. Due to the complexity and NP-hardness of the formulated TEMCMOP, we propose an improved multi-objective ant lion optimization (IMOALO) algorithm with chaos-opposition based learning solution initialization and hybrid solution update operators to solve the problem. Simulation results verify that the proposed IMOALO can effectively solve the formulated TEMCMOP and it has better performance than some other benchmark approaches. [ABSTRACT FROM AUTHOR]

In mobile crowdsensing, finding the best match between tasks and users is crucial to ensure both the quality and effectiveness of a crowdsensing system. Existing works usually assume a centralized task assignment by the crowdsensing platform, without addressing the need of fine-grained personalized task matching. In this paper, we argue that it is essential to match tasks to users based on a careful characterization of both the users’ preference and reliability. To that end, we propose a personalized task recommender system for mobile crowdsensing, which recommends tasks to users based on a recommendation score that jointly takes each user’s preference and reliability into consideration. We first present a hybrid preference metric to characterize users’ preference by exploiting their implicit feedback. Then, to profile users’ reliability levels, we formalize the problem as a semi-supervised learning model, and propose an efficient block coordinate descent algorithm to solve the problem. For some tasks that lack users’ historical information, we further propose a matrix factorization method to infer the users’ reliability levels on those tasks. We conduct extensive experiments to evaluate the performance of our system, and the evaluation results demonstrate that our system can achieve superior performance to the benchmarks in both user profiling and personalized task recommendation. [ABSTRACT FROM AUTHOR]

In the task of person re-identification, the attention mechanism and fine-grained information have been proved to be effective. However, it has been observed that models often focus on the extraction of features with strong discrimination, and neglect other valuable features. The extracted fine-grained information may include redundancies. In addition, current methods lack an effective scheme to remove background interference. Therefore, this paper proposes the feature refinement and filter network to solve the above problems from three aspects: first, by weakening the high response features, we aim to identify highly valuable features and extract the complete features of persons, thereby enhancing the robustness of the model; second, by positioning and intercepting the high response areas of persons, we eliminate the interference arising from background information and strengthen the response of the model to the complete features of persons; finally, valuable fine-grained features are selected using a multi-branch attention network for person re-identification to enhance the performance of the model. Our extensive experiments on the benchmark Market-1501, DukeMTMC-reID, CUHK03 and MSMT17 person re-identification datasets demonstrate that the performance of our method is comparable to that of state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Performing deep neural network (DNN) inference in real time requires excessive network resources, which poses a big challenge to the resource-limited industrial Internet of things (IIoT) networks. To address the challenge, in this paper, we introduce an end-edge-cloud orchestration architecture, in which the inference task assignment and DNN model placement are flexibly coordinated. Specifically, the DNN models, trained and pre-stored in the cloud, are properly placed at the end and edge to perform DNN inference. To achieve efficient DNN inference, a multi-dimensional resource management problem is formulated to maximize the average inference accuracy while satisfying the strict delay requirements of inference tasks. Due to the mix-integer decision variables, it is difficult to solve the formulated problem directly. Thus, we transform the formulated problem into a Markov decision process which can be solved efficiently. Furthermore, a deep reinforcement learning based resource management scheme is proposed to make real-time optimal resource allocation decisions. Simulation results are provided to demonstrate that the proposed scheme can efficiently allocate the available spectrum, caching, and computing resources, and improve average inference accuracy by 31.4 % compared with the deep deterministic policy gradient benchmark. [ABSTRACT FROM AUTHOR]

Learning visual semantic embedding for image-text matching has achieved high success by using triplet loss to pull positive image-text pairs which share similar semantic meaning and to push negative image-text pairs which share different semantic meaning. Without modeling constraints from image-image or text-text pairs, the generated visual semantic embedding inevitably faces the problem of semantic misalignments among similar images or among similar texts. To solve this problem, we present a contrastive visual semantic embedding framework, named ConVSE, which achieves intra-modal semantic alignment by contrastive learning from augmented image-image (or text-text) pairs and achieves inter-modal semantic alignment by applying hardest-negative-enhanced triplet loss on image-text pairs. To the best of our knowledge, we are the first to find that contrastive learning benefits visual semantic embedding. Extensive experiments on large-scale MSCOCO and Flickr30 K datasets verify the effectiveness of our proposed ConVSE by outperforming visual semantic embedding-based methods and achieving new state-of-the-art. [ABSTRACT FROM AUTHOR]

We compare physical and virtual reality (VR) versions of simple data visualizations and explore how the addition of virtual annotation and filtering tools affects how viewers solve basic data analysis tasks. We report on two studies, inspired by previous examinations of data physicalizations. The first study examines differences in how viewers interact with physical hand-scale, virtual hand-scale, and virtual table-scale visualizations and the impact that the different forms had on viewer’s problem solving behavior. A second study examines how interactive annotation and filtering tools might support new modes of use that transcend the limitations of physical representations. Our results highlight challenges associated with virtual reality representations and hint at the potential of interactive annotation and filtering tools in VR visualizations. [ABSTRACT FROM AUTHOR]

Evolutionary algorithms (EAs) possess strong problem-solving abilities and have been applied in a wide range of applications. However, they still suffer from a high computational burden and poor generalization ability. To overcome the limitations, numerous studies consider conducting knowledge extraction across distinct optimization task domains. Among these research strands, one representative tributary is evolutionary multitask optimization (EMTO) that aims to resolve multiple optimization tasks simultaneously. The underlying attribute of implicit parallelism for EAs can well incorporate with the framework of EMTO, giving rise to the ascending EMTO studies. This review is intended to present a detailed exposition on the research in the EMTO area. We reveal the core components for designing the EMTO algorithms. Subsequently, we organize the works lying in the fusions between EMTO and traditional EAs. By analyzing the associations for diverse strategies in different branches of EMTO, this review uncovers the research trends and the potentially important directions, with additional interesting real-world applications mentioned. [ABSTRACT FROM AUTHOR]

Although transformer has achieved great progress on computer vision tasks, the scale variation in dense image prediction is still the key challenge. Few effective multi-scale techniques are applied in transformer and there are two main limitations in the current methods. On the one hand, self-attention module in vanilla transformer fails to sufficiently exploit the diversity of semantic information because of its rigid mechanism. On the other hand, it is difficult to build attention and interaction among different levels due to the heavy computational burden. To alleviate this problem, we first revisit multi-scale problem in dense prediction, verifying the significance of diverse semantic representation and multi-scale interaction, and exploring the adaptation of transformer to pyramidal structure. Inspired by these findings, we propose a novel Semantic-aware Decoupled Transformer Pyramid (SDTP) for dense image prediction, consisting of Intra-level Semantic Promotion (ISP), Cross-level Decoupled Interaction (CDI) and Attention Refinement Function (ARF). ISP explores the semantic diversity in different receptive space through more flexible self-attention strategy. CDI builds the global attention and interaction among different levels in decoupled space which also solves the problem of heavy computation. Besides, ARF is further added to refine the attention in transformer. Experimental results demonstrate the validity and generality of the proposed method, which outperforms the state-of-the-art by a significant margin in dense image prediction tasks. Furthermore, the proposed components are all plug-and-play, which can be embedded in other methods. [ABSTRACT FROM AUTHOR]

This article focuses on solving the problem of precise peg-in-hole assembly. The main contribution of this article is to propose a control method to improve the assembly precision. The method is capable to realize the precise insertion assembly tasks of cylindrical or prismatic objects with clearance or interference fit. In this article, a feature space and a response space are first constructed according to the relative pose and equivalent forces and moments. Then, the contact states are segmented in the feature space and the segmentation boundaries are mapped into the response space. Furthermore, a feature-based compliance control (FBCC) method is proposed based on boundary mapping. In FBCC method, a direction matrix is designed to execute accurate adjustment and an integral element is applied to eliminate residual responses. Finally, the simulations and experiments demonstrate the superiority and robustness of FBCC method. [ABSTRACT FROM AUTHOR]

Knowledge transfer plays a vastly important role in solving multitask optimization problems (MTOPs). Many existing methods transfer task-specific knowledge, such as the high-quality solution from one task to other tasks to enhance the optimization ability, which, however, may not work well or even have a negative effect if the tasks have very different task-specific knowledge. Hence, this article proposes a meta-knowledge transfer (MKT)-based differential evolution (MKTDE) algorithm by using a more general MKT method to solve MTOPs more efficiently. The meta-knowledge defined in this article refers to the knowledge that can evolve task-specific knowledge during the evolutionary search. That is, the meta-knowledge is a kind of “knowledge of knowledge,” which denotes the knowledge of “how to solve problem via evolution” and “the feature/way/method of evolving high-quality solution.” The evolutionary search for solving different tasks can share common meta-knowledge even though these tasks involve heterogeneous data and have very different task-specific knowledge. Therefore, the MKT can associate the heterogeneous multisource data of different tasks via transferring the meta-knowledge to help solve MTOPs more efficiently in a more general way. Moreover, to further enhance the MKTDE, two novel and efficient methods are proposed. One is multiple populations for the multiple tasks framework using a unified search space for making knowledge transfer flexibly. The other is an elite solution transfer method for achieving positive high-quality solution transfer. The superior performance of the proposed MKTDE is verified via extensive numerical experiments on both widely used MTOP benchmark problems and real-world robot navigation problems, with comparisons with some state-of-the-art and the latest well-performing algorithms. [ABSTRACT FROM AUTHOR]

Collaborative multiagent robotic systems, where agents coordinate by modifying a shared environment often result in undesired dynamical couplings that complicate the analysis and experiments when solving a specific problem or task. Simultaneously, biologically inspired robotics rely on simplifying agents and increasing their number to obtain more efficient solutions to such problems, drawing similarities with natural processes. In this work, we focus on the problem of a biologically inspired multiagent system solving collaborative foraging. We show how mean field techniques can be used to re-formulate such a stochastic multiagent problem into a deterministic autonomous system. This de-couples agent dynamics, enabling the computation of limit behaviors and the analysis of optimality guarantees. Furthermore, we analyse how having finite number of agents affects the performance when compared to the mean field limit and we discuss the implications of such limit approximations in this multiagent system, which have impact on more general collaborative stochastic problems. [ABSTRACT FROM AUTHOR]

Video caption aims to generate descriptive sentences about the video, and the most critical problem is how to achieve accurate word prediction with standardized and coherent syntax structure, which requires the model to thoroughly understand video content and precisely map them into corresponding sentence components. Many existing methods usually fuse different video features into a single visual feature for generating sentences. However, they ignore the word dataset prior information in the annotations (such as Part-Of-Speech) and they also ignore the association between sentence components and types of visual features. To solve these problems, we propose a POS-trends dynamic-aware model (PDA) to fully exploit the word dataset prior information in the captions to predict POS tag, so as to assist generating captions. We propose a POS feature extraction (PFE) module to use different filters to extract different POS-trends features, predict POS tags and fuse visual features. Furthermore, we propose a visual-dynamic-aware (VDA) module to dynamically adjust the mapping way of words and supplement the visual information into the local features. The fusion features provide directional visual information to generate correct words, and the predicted POS tags to guide the decoding process to generate a more standardized and coherent syntax structure. A large number of experiments based on MSVD, MSR-VTT and VATEX demonstrated that our method outperforms the state-of-the-art methods in BLEU-4, ROUGE-L, METEOR, CIDEr. Code can be available at: https://github.com/WangLanxiao/PDA-for-video-caption. [ABSTRACT FROM AUTHOR]

Hyperspectral anomaly detection (AD) aims to detect objects significantly different from their surrounding background. Recently, many detectors based on autoencoder (AE) exhibited promising performances in hyperspectral AD tasks. However, the fundamental hypothesis of the AE-based detector that anomaly is more challenging to be reconstructed than background may not always be true in practice. We demonstrate that an AE could well reconstruct anomalies even without anomalies for training, because AE models mainly focus on the quality of sample reconstruction and do not care if the encoded features solely represent the background rather than anomalies. If more information is preserved than needed to reconstruct the background, the anomalies will be well reconstructed. This article proposes a cluster-memory augmented deep autoencoder via optimal transportation for hyperspectral anomaly detection (OTCMA) clustering for hyperspectral AD to solve this problem. The deep clustering method based on optimal transportation (OT) is proposed to enhance the features consistency of samples within the same categories and features discrimination of samples in different categories. The memory module stores the background’s consistent features, which are the cluster centers for each category background. We retrieve more consistent features from the memory module instead of reconstructing a sample utilizing its own encoded features. The network focuses more on consistent feature reconstruction by training AE with a memory module. This effectively restricts the reconstruction ability of AE and prevents reconstructing anomalies. Extensive experiments on the benchmark datasets demonstrate that our proposed OTCMA achieves state-of-the-art results. Besides, this article presents further discussions about the effectiveness of our proposed memory module and different criteria for better AD. [ABSTRACT FROM AUTHOR]

Since the superpixel segmentation method aggregates pixels based on similarity, the boundaries of some superpixels indicate the outline of the object and the superpixels provide prerequisites for learning structural-aware features. It is worthwhile to research how to utilize these superpixel priors effectively. In this work, by constructing the graph within superpixel and the graph among superpixels, we propose a novel Multi-level Feature Network (MFNet) based on graph neural network with the above superpixel priors. In our MFNet, we learn three-level features in a hierarchical way: from pixel-level feature to superpixel-level feature, and then to image-level feature. To solve the problem that the existing methods cannot represent superpixels well, we propose a superpixel representation method based on graph neural network, which takes the graph constructed by a single superpixel as input to extract the feature of the superpixel. To reflect the versatility of our MFNet, we apply it to an image-level prediction task and a pixel-level prediction task by designing different prediction modules. An attention linear classifier prediction module is proposed for image-level prediction tasks, such as image classification. An FC-based superpixel prediction module and a Decoder-based pixel prediction module are proposed for pixel-level prediction tasks, such as salient object detection. Our MFNet achieves competitive results on a number of datasets when compared with related methods. The visualization shows that the object boundaries and outline of the saliency maps predicted by our proposed MFNet are more refined and pay more attention to details. [ABSTRACT FROM AUTHOR]

Unsupervised person re-identification (Re-Id) has attracted increasing attention due to its practical application in the read-world video surveillance system. The traditional unsupervised Re-Id are mostly based on the method alternating between clustering and fine-tuning with the classification or metric learning objectives on the grouped clusters. However, since person Re-Id is an open-set problem, the clustering based methods often leave out lots of outlier instances or group the instances into the wrong clusters, thus they can not make full use of the training samples as a whole. To solve these problems, we present the hybrid dynamic cluster contrast and probability distillation algorithm. It formulates the unsupervised Re-Id problem into an unified local-to-global dynamic contrastive learning and self-supervised probability distillation framework. Specifically, the proposed method can make the best of the self-supervised signals of all the clustered and un-clustered instances, from both the instances’ self-contrastive level and the probability distillation respectives, in the memory-based non-parametric manner. Besides, the proposed hybrid local-to-global contrastive learning can take full advantage of the informative and valuable training examples for effective and robust training. Extensive experiment results show that the proposed method achieves superior performances to state-of-the-art methods, under both the purely unsupervised and unsupervised domain adaptation experiment settings. Our source code is released in https://github.com/zjy2050/HDCRL-ReID. [ABSTRACT FROM AUTHOR]

Online social media have aroused much research interest in recent years. In contrast to previous work that focused on the detection of emerging topics, this article undertakes the prediction of active users in online social events, which is so far rarely explored. This prediction task is formulated as a binary classification problem that built on real-world tweet streams, taking Ferguson event and New York Chockhold event as examples. Then, a comprehensive user feature system is designed to characterize the events’ online participants, which includes not only basic statistical characteristics and image-pixel-level features, but also some emotional features and personality features. Next, the Weighted Random Forest (Weighted-RF) classifier is adopted to solve the classification problem. Based on the user feature system and the classifier, the experience of a previous event can be archived and applied to the prediction of later similar events. Experimental results show that the Weighted-RF trained by samples of Ferguson event can effectively predict active users in NYC event, with an AUC value around 0.8392. Besides, the image-content based personality model provides a new tool for depicting user portraits, which further contributes to the quantitative analysis of online social events. [ABSTRACT FROM AUTHOR]

Multitask modeling methods for Takagi–Sugeno–Kang (TSK) fuzzy systems exhibit better generalization ability attributed to the utilization of the knowledge of intertask correlation. However, existing methods usually ignore the balance between the sharing of the common knowledge across multiple tasks and the preservation of the task-specific characteristics of each rule. To this end, we propose a novel manifold-regularized multitask modeling method for TSK fuzzy system by introducing low-rank and sparse structures into consequent parameters across multiple tasks. Specifically, we decompose the consequent parameters into two components—a task–shared component that represents similar structure across multiple tasks, and a task-specific component that encodes the sparse characteristics of the individual tasks. This can be implemented by imposing low-rank constraints on the task-shared component and applying the sparse constraints on the task-specific component. A new manifold regularization is further devised to reflect the feature-feature relation, which provides prior knowledge in multitask learning. An efficient augmented Lagrange multiplier is developed to solve the optimization problem. The experimental results demonstrate that the proposed model significantly outperforms the existing methods. [ABSTRACT FROM AUTHOR]

A heterogeneous information network (HIN) is one whose nodes model objects of different types and whose links model objects’ relationships. To enrich its information, objects in an HIN are typically associated with additional attributes. We call such an HIN an Attributed HIN or AHIN. We study the problem of clustering objects in an AHIN, taking into account objects’ similarities with respect to both object attribute values and their structural connectedness in the network. We show how supervision signal, expressed in the form of a must-link set and a cannot-link set, can be leveraged to improve clustering results. We put forward the SCHAIN algorithm to solve the clustering problem, and two highly efficient variants, SCHAIN-PI and SCHAIN-IRAM, which employ the power iteration based method and the implicitly restarted Arnoldi method respectively to compute eigenvectors of a matrix. We conduct extensive experiments comparing SCHAIN-based algorithms with other state-of-the-art clustering algorithms. Our results show that SCHAIN-IRAM outperforms other competitors in terms of clustering effectiveness and is highly efficient. [ABSTRACT FROM AUTHOR]

Irrefutable evidence shows that greater openness toward external partners enhances a firm's ability to solve innovation-related problems. To manage open innovation (OI) projects, firms use a variety of governance modes, including market-based contracts, platform intermediaries, and equity and non-equity partnerships. While innovation projects can be very diverse and characterized by various attributes, such as complexity and knowledge hiddenness, only a few conceptual studies have hitherto considered project attributes as drivers of OI governance mode selection. Using a sample of 85 OI projects and a set of illustrative cases, this article explores empirically how project attributes influence OI governance mode selection. This empirical study advances previous conceptual work on OI governance. By accounting for project-level heterogeneity, we explore the microfoundations of OI and provide more stable and fundamental insights into OI governance than previous industry- and firm-level analyses did. In addition, we suggest that effective OI management depends on matching project attributes with the benefits and costs of specific governance modes. Finally, we argue that this study enhances understanding and conceptualization of the relationship between project complexity and decomposability in the context of OI. [ABSTRACT FROM AUTHOR]

Analyzing short texts infers discriminative and coherent latent topics that is a critical and fundamental task since many real-world applications require semantic understanding of short texts. Traditional long text topic modeling algorithms (e.g., PLSA and LDA) based on word co-occurrences cannot solve this problem very well since only very limited word co-occurrence information is available in short texts. Therefore, short text topic modeling has already attracted much attention from the machine learning research community in recent years, which aims at overcoming the problem of sparseness in short texts. In this survey, we conduct a comprehensive review of various short text topic modeling techniques proposed in the literature. We present three categories of methods based on Dirichlet multinomial mixture, global word co-occurrences, and self-aggregation, with example of representative approaches in each category and analysis of their performance on various tasks. We develop the first comprehensive open-source library, called STTM, for use in Java that integrates all surveyed algorithms within a unified interface, benchmark datasets, to facilitate the expansion of new methods in this research field. Finally, we evaluate these state-of-the-art methods on many real-world datasets and compare their performance against one another and versus long text topic modeling algorithm. [ABSTRACT FROM AUTHOR]

The encoder–decoder framework is prevalent in existing remote-sensing image captioning (RSIC) models. The appearance of attention mechanisms brings significant results. However, current attention-based caption models only build up the relationships between the local features without introducing the global visual feature and removing redundant feature components. It will cause caption models to generate descriptive sentences that are weakly related to the scene of images. To solve the problems, this article proposed a global visual feature-guided attention (GVFGA) mechanism. First, GVFGA introduces the global visual feature and fuses them with local visual features to build up their relationships between them. Second, an attention gate utilizing the global visual feature is proposed in GVFGA to filter out redundant feature components in the fused image features and provide more salient image features. In addition, to relieve the hidden state’s burden, a linguistic state (LS) is proposed to specifically provide textual features, making the hidden state only guiding visual–textual attention process. What’s more, to further refine the fusion of visual features and textual features, a LS-Guided Attention (LSGA) mechanism is proposed. It can also filter out the irrelevant information in the fused visual–textual feature with the help of an attention gate. The experimental results show that this proposed image captioning model can achieve better results on three RSIC datasets, UCM-Captions, Sydney-Captions, and RSICD datasets. [ABSTRACT FROM AUTHOR]

As a key mission of the modern traffic management, crowd flow prediction (CFP) benefits in many tasks of intelligent transportation services. However, most existing techniques focus solely on forecasting entrance and exit flows of metro stations that do not provide enough useful knowledge for traffic management. In practical applications, managers desperately want to solve the problem of getting the potential passenger distributions to help authorities improve transport services, termed as crowd flow distribution (CFD) forecasts. Therefore, to improve the quality of transportation services, we proposed three spatiotemporal models to effectively address the network-wide CFD prediction problem based on the online latent space (OLS) strategy. Our models take into account the various trending patterns and climate influences, as well as the inherent similarities among different stations that are able to predict both CFD and entrance and exit flows precisely. In our online systems, a sequence of CFD snapshots is used as the training data. The latent attribute evolutions of different metro stations can be learned from the previous trend and do the next prediction based on the transition patterns. All the empirical results demonstrate that the three developed models outperform all the other state-of-the-art approaches on three large-scale real-world datasets. [ABSTRACT FROM AUTHOR]

Given the prior information of the target, hyperspectral target detection focuses on exploiting spectral differences to separate objects of interest from the background, which can be treated as information retrieval (IR) task in machine learning (ML). Most traditional detection methods work in the original feature space and rely heavily on specific assumptions, which cannot guarantee effective extraction of features for the target and background in hyperspectral images (HSIs). Mass estimation (ME) is a base modeling mechanism that has been proven to effectively solve problems in IR and is not restricted by specific assumptions. In this article, we propose a novel target detection method through ensemble-based IR with ME (EIRME). By directly deriving the ordering from a sample set to rank data points, ME provides a simple and straightforward ranking measure to ensure that points similar to the given target are far away from dissimilar points. For the estimation of mass distribution, the proposed method utilizes a tree-structured mapping to generate a feature space, in which the separability of the target and background is further improved. In particular, to break through the technical difficulty that the direct migration of IR methods with mass measure cannot specifically meet the high-precision requirements of target detection in HSIs, we develop a specialized measurement, topological mass, which innovatively combines the mass measure with tree topology to quantify the spectral difference for detection output. Moreover, the IR with ME based on parallel measurements through ensemble trees provides a robust solution with better generalization capacity and higher precision for hyperspectral target detection, facilitating practical applications. Experimental results on benchmark HSI datasets prove that the specialized measurement that we developed successfully overcomes the drawbacks of the direct migration of IR methods with ME and exhibits unique advantages. In addition, comparisons with the most classic and advanced detection algorithms demonstrate the superiority of the proposed method. [ABSTRACT FROM AUTHOR]

Cohort selection is an essential prerequisite for clinical research, determining whether an individual satisfies given selection criteria. Previous works for cohort selection usually treated each selection criterion independently and ignored not only the meaning of each selection criterion but the relations among cohort selection criteria. To solve the problems above, we propose a novel unified machine reading comprehension (MRC) framework. In this MRC framework, we design simple rules to generate questions for each criterion from cohort selection guidelines and treat clues extracted by trigger words from patients' medical records as passages. A series of state-of-the-art MRC models based on BiDAF, BIMPM, BERT, BioBERT, NCBI-BERT, and RoBERTa are deployed to determine which question and passage pairs match. We also introduce a cross-criterion attention mechanism on representations of question and passage pairs to model relations among cohort selection criteria. Results on two datasets, that is, the dataset of the 2018 National NLP Clinical Challenge (N2C2) for cohort selection and a dataset from the MIMIC-III dataset, show that our NCBI-BERT MRC model with cross-criterion attention mechanism achieves the highest micro-averaged F1-score of 0.9070 on the N2C2 dataset and 0.8353 on the MIMIC-III dataset. It is competitive to the best system that relies on a large number of rules defined by medical experts on the N2C2 dataset. Comparing these two models, we find that the NCBI-BERT MRC model mainly performs worse on mathematical logic criteria. When using rules instead of the NCBI-BERT MRC model on some criteria regarding mathematical logic on the N2C2 dataset, we obtain a new benchmark with an F1-score of 0.9163, indicating that it is easy to integrate rules into MRC models for improvement. [ABSTRACT FROM AUTHOR]

Visual Question Answering (VQA) is a task to answer natural language questions tied to the content of visual images. Most recent VQA approaches usually apply attention mechanism to focus on the relevant visual objects and/or consider the relations between objects via off-the-shelf methods in visual relation reasoning. However, they still suffer from several drawbacks. First, they mostly model the simple relations between objects, which results in many complicated questions cannot be answered correctly, because of failing to provide sufficient knowledge. Second, they seldom leverage the harmony cooperation of visual appearance feature and relation feature. To solve these problems, we propose a novel end-to-end VQA model, termed Multi-modal Relation Attention Network (MRA-Net). The proposed model explores both textual and visual relations to improve performance and interpretability. In specific, we devise 1) a self-guided word relation attention scheme, which explore the latent semantic relations between words; 2) two question-adaptive visual relation attention modules that can extract not only the fine-grained and precise binary relations between objects but also the more sophisticated trinary relations. Both kinds of question-related visual relations provide more and deeper visual semantics, thereby improving the visual reasoning ability of question answering. Furthermore, the proposed model also combines appearance feature with relation feature to reconcile the two types of features effectively. Extensive experiments on five large benchmark datasets, VQA-1.0, VQA-2.0, COCO-QA, VQA-CP v2, and TDIUC, demonstrate that our proposed model outperforms state-of-the-art approaches. [ABSTRACT FROM AUTHOR]