Search

Multi-Task Learning (MTL) is a learning paradigm in machine learning and its aim is to leverage useful information contained in multiple related tasks to help improve the generalization performance of all the tasks. In this paper, we give a survey for MTL from the perspective of algorithmic modeling, applications and theoretical analyses. For algorithmic modeling, we give a definition of MTL and then classify different MTL algorithms into five categories, including feature learning approach, low-rank approach, task clustering approach, task relation learning approach and decomposition approach as well as discussing the characteristics of each approach. In order to improve the performance of learning tasks further, MTL can be combined with other learning paradigms including semi-supervised learning, active learning, unsupervised learning, reinforcement learning, multi-view learning and graphical models. When the number of tasks is large or the data dimensionality is high, we review online, parallel and distributed MTL models as well as dimensionality reduction and feature hashing to reveal their computational and storage advantages. Many real-world applications use MTL to boost their performance and we review representative works in this paper. Finally, we present theoretical analyses and discuss several future directions for MTL. [ABSTRACT FROM AUTHOR]

Benefiting from the rapid progress of wireless communication and artificial intelligence, multi-agent collaboration opens up new opportunities for various fields. To facilitate multi-agent acting as a group, effective communication plays a crucial role. Recently, many efforts based on multi-agent reinforcement learning have been made to enable effective multi-agent communication under limited bandwidth or noisy channel. However, current methods do not explore wireless resource allocation strategy explicitly. Moreover, due to ignoring task-relevant significance of information, traditional wireless resource allocation schemes may fail to guarantee the transmission efficiency and reliability for multi-agent communication. To this end, in this paper, we propose a task-oriented communication principle for multi-agent communication. We model the task-oriented channel allocation problem as a decentralized partially observable Markov decision process and propose a multi-agent reinforcement learning framework as a solution. Specifically, we design a novel variational information bottleneck to extract task-relevant information from local observation. Furthermore, a task-oriented channel allocation mechanism is developed to choose the allocation pattern with maximum expected gain. Finally, a double attention mechanism is developed to motivate the efficient utilization of task-relevant information. Experimental results show that our method can improve the effectiveness and efficiency of multi-agent communication, enhancing collaboration performance compared to baselines. [ABSTRACT FROM AUTHOR]

Low level image restoration is an integral component of modern artificial intelligence (AI) driven camera pipelines. Most of these frameworks are based on deep neural networks which present a massive computational overhead on resource constrained platform like a mobile phone. In this paper, we propose several lightweight low-level modules which can be used to create a computationally low cost variant of a given baseline model. Recent works for efficient neural networks design have mainly focused on classification. However, low-level image processing falls under the ‘image-to-image’ translation genre which requires some additional computational modules not present in classification. This paper seeks to bridge this gap by designing generic efficient modules which can replace essential components used in contemporary deep learning based image restoration networks. We also present and analyse our results highlighting the drawbacks of applying depthwise separable convolutional kernel (a popular method for efficient classification network) for sub-pixel convolution based upsampling (a popular upsampling strategy for low-level vision applications). This shows that concepts from domain of classification cannot always be seamlessly integrated into ‘image-to-image’ translation tasks. We extensively validate our findings on three popular tasks of image inpainting, denoising and super-resolution. Our results show that proposed networks consistently output visually similar reconstructions compared to full capacity baselines with significant reduction of parameters, memory footprint and execution speeds on contemporary mobile devices. [ABSTRACT FROM AUTHOR]

Artificial intelligence (AI)-powered autonomous vehicles (AVs) can integrate different machine learning (ML) techniques to build up a complex autonomous driving system. However, single AV intelligence is not enough to cope with ever-changing driving environments. The underlying reason is that, with current neural network design and training algorithms, it is challenging for the driving model to generalize to diverse driving environments all at once due to sample inefficiency and the curse of dimensionality. Powerful computing resources and massive amount of data can be used to train a good driving model offline. However, the driving model obtained offline might fail in corner case scenarios. In this paper, we propose an intelligence networking framework among AVs assisted by multi-access edge computing (MEC) with end-to-end learning for demonstration. In this framework, driving road is divided into segments and data is collected for each road segment separately. Assisted by MEC networks, a continuously updated driving model is produced in near real-time for each road segment when the environment changes. By dividing the road into segments, we aim to reduce the burden of generalization since a single model only needs to adapt to a specific road segment. Simulation results show that our solutions can produce updated driving model for each road segment to adapt to environmental changes better than the existing scheme. Upcoming AVs can then adapt to changing environments by downloading the updated driving model. [ABSTRACT FROM AUTHOR]

Crowdsourcing is a hotspot research field which can facilitate machine learning by collecting labels to train models. Consequently, the state-of-the-art research efforts in crowdsourcing focus on truth inference or label integration, to remove inconsistent labels or to alleviate biased labeling. In turn, the integrated labels will be used to fine-tune machine learning models. Particularly, in this paper, we change the target of truth inference in crowdsourcing from discrete labels to multiple comments given by online participants, that is, the integration of the crowdsourced comments. For such a goal, we propose a Self-play and Sentiment-Emphasized Comment Integration Framework (SSECIF), based on deep Q-learning, with three unique features. First, our framework SSECIF can generate the comment integration in a totally self-play way, without relying on the ground truth generated by human effort. Second, the integrated comment generated by SSECIF can include salient content with low redundancy. Third, the proposed framework SSECIF has emphasized, with a higher intensity, the sentiment in the integrated comment, in order to reflect the attitude or opinion more obviously. Extensive evaluation on real-world datasets demonstrates that SSECIF has achieved the best overall performance in terms of both effectiveness and efficiency, compared with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Portfolio Selection is an important real-world financial task and has attracted extensive attention in artificial intelligence communities. This task, however, has two main difficulties: (i) the non-stationary price series and complex asset correlations make the learning of feature representation very hard; (ii) the practicality principle in financial markets requires controlling both transaction and risk costs. Most existing methods adopt handcraft features and/or consider no constraints for the costs, which may make them perform unsatisfactorily and fail to control both costs in practice. In this paper, we propose a cost-sensitive portfolio selection method with deep reinforcement learning. Specifically, a novel two-stream portfolio policy network is devised to extract both price series patterns and asset correlations, while a new cost-sensitive reward function is developed to maximize the accumulated return and constrain both costs via reinforcement learning. We theoretically analyze the near-optimality of the proposed reward, which shows that the growth rate of the policy regarding this reward function can approach the theoretical optimum. We also empirically evaluate the proposed method on real-world datasets. Promising results demonstrate the effectiveness and superiority of the proposed method in terms of profitability, cost-sensitivity and representation abilities. [ABSTRACT FROM AUTHOR]

Recently, unmanned aerial vehicle (UAV)-enabled multi-access edge computing (MEC) has been introduced as a promising edge paradigm for the future space-aerial-terrestrial integrated communications. Due to the high maneuverability of UAVs, such a flexible paradigm can improve the communication and computation performance for multiple user equipments (UEs). In this paper, we consider the sum power minimization problem by jointly optimizing resource allocation, user association, and power control in an MEC system with multiple UAVs. Since the problem is nonconvex, we propose a centralized multi-agent reinforcement learning (MARL) algorithm to solve it. However, the centralized method ignores essential issues like distributed framework and privacy concern. We then propose a multi-agent federated reinforcement learning (MAFRL) algorithm in a semi-distributed framework. Meanwhile, we introduce the Gaussian differentials to protect the privacy of all UEs. Simulation results show that the semi-distributed MAFRL algorithm achieves close performances to the centralized MARL algorithm and significantly outperform the benchmark schemes. Moreover, the semi-distributed MAFRL algorithm costs 23 $\%$ lower opeartion time than the centralized algorithm. [ABSTRACT FROM AUTHOR]

Traditional Artificial Intelligence (AI) based fault detection approaches need a large amount of data for the model learning. However, in a real-world system, it is very difficult and expensive to obtain massive labeled fault data. In addition, the working conditions of a motor are usually variable, conventional fault diagnosis models with weak generalization ability can only be used for fault detection under constant working condition. The performance of traditional AI based approaches decreases when the working condition changes. To this end, a novel deep Gaussian process (GP) kernel transfer based few-shot learning method (RNGPT) is proposed in this paper for the fault detection of electric machines. First, a deep residual network (ResNet) is used to extract the features of the raw data. Then, the encoded latent feature vector is fed into the GP with kernel transfer ability to make the motor fault detection and classification. The proposed method uses much less data than the traditional AI based method to achieve fault diagnosis under variable working condition, and does not cause an overfitting problem. Experimental results of two case studies demonstrate that the proposed RNGPT model can accurately and effectively detect motor faults with limited labeled data under different working conditions. Experimental results of RNGPT with radial basis function (RBF) kernel model on simulation data present that the fault detection accuracy of the proposed method is about 16% higher than the conventional deep learning methods, 6% higher than other few-shot learning based methods in 5-shot and 4% higher in 1-shot. Finally, experimental on a real-world dataset, the RNGPT-RBF model still has the highest fault diagnosis accuracy in 5-shot (99.39 $ \pm $ 0.09%) and 1-shot (98.55 $ \pm $ 0.16%). [ABSTRACT FROM AUTHOR]

Recently neural architecture search (NAS) has raised great interest in both academia and industry. However, it remains challenging because of its huge and non-continuous search space. Instead of applying evolutionary algorithm or reinforcement learning as previous works, this paper proposes a direct sparse optimization NAS (DSO-NAS) method. The motivation behind DSO-NAS is to address the task in the view of model pruning. To achieve this goal, we start from a completely connected block, and then introduce scaling factors to scale the information flow between operations. Next, sparse regularizations are imposed to prune useless connections in the architecture. Lastly, an efficient and theoretically sound optimization method is derived to solve it. Our method enjoys both advantages of differentiability and efficiency, therefore it can be directly applied to large datasets like ImageNet and tasks beyond classification. Particularly, on the CIFAR-10 dataset, DSO-NAS achieves an average test error 2.74 percent, while on the ImageNet dataset DSO-NAS achieves 25.4 percent test error under 600M FLOPs with 8 GPUs in 18 hours. As for semantic segmentation task, DSO-NAS also achieve competitive result compared with manually designed architectures on the PASCAL VOC dataset. Code is available at https://github.com/XinbangZhang/DSO-NAS. [ABSTRACT FROM AUTHOR]

Images captured in snowy days suffer from noticeable degradation of scene visibility, which degenerates the performance of current vision-based intelligent systems. Removing snow from images thus is an important topic in computer vision. In this paper, we propose a Deep Dense Multi-Scale Network (DDMSNet) for snow removal by exploiting semantic and depth priors. As images captured in outdoor often share similar scenes and their visibility varies with depth from camera, such semantic and depth information provides a strong prior for snowy image restoration. We incorporate the semantic and depth maps as input and learn the semantic-aware and geometry-aware representation to remove snow. In particular, we first create a coarse network to remove snow from the input images. Then, the coarsely desnowed images are fed into another network to obtain the semantic and depth labels. Finally, we design a DDMSNet to learn semantic-aware and geometry-aware representation via a self-attention mechanism to produce the final clean images. Experiments evaluated on public synthetic and real-world snowy images verify the superiority of the proposed method, offering better results both quantitatively and qualitatively. https://github.com/HDCVLab/Deep-Dense-Multi-scale-Network https://github.com/HDCVLab/Deep-Dense-Multi-scale-Network. [ABSTRACT FROM AUTHOR]

With the development of artificial intelligence(AI), unmanned vehicles can relieve traffic jamming and decrease the risk of traffic accidents, where deep neural networks (DNNs) play an important role and have become one of the most critical technologies. Nevertheless, DNNs are still susceptible to adversarial examples. Even worse, they also show severe performance degradation when the system needs DNNs to learn new knowledge without forgetting the old one. As unmanned vehicles travel on the road, they need to frequently learn new categories and different representations. Learning all data after the new sample arrives will expend a lot of time and space. As a result, it will affect the deployment of artificial intelligence in unmanned scenes. In recent years, it has been observed that incremental learning technology can solve the above challenges. However, previously reported works mainly focused on batch learning. It is not clear how much impact the adversarial attack will have on the deep learning model when performing incremental learning tasks. This issue exposes the hidden safety risks of unmanned driving and increases discuss opportunities. Therefore, we propose an adversarial attack based on incremental learning techniques for unmanned scenes in this paper. Specifically, it can retain information previously learned by the model. At the same time, it can renew the old model to learn new model, thereby continually adding small perturbation to legitimate examples. A couple of experiments on the Pascal VOC 2012 dataset has been conducted, and the experiment results show that the adversarial attack based on incremental learning techniques has a higher attack success rate. Further, it can improve the successful attack rate by 8.43%. [ABSTRACT FROM AUTHOR]

Robotic devices can be leveraged to raise the abilities of humans to perform demanding and complex tasks with less effort. Although the first priority of such human–robot interaction (HRI) is safety, robotic devices must also be intuitive and efficient in order to be adopted by a broad range of users. One challenge in the control of such assistive robots is the management of the end-effector orientation, that is not always intuitive for the human operator, especially for neophytes. This paper presents a novel orientation control algorithm designed for robotic arms in the context of HRI. This paper aims at making the control of the robot's orientation easier and more intuitive for the user, both in the fields of rehabilitation (in particular individuals living with upper limb disabilities) and industrial robotics. The performance and intuitiveness of the proposed orientation control algorithm is assessed and improved through two experiments with a JACO assistive robot with 25 able-bodied subjects, an online survey with 117 respondents via the Amazon Mechanical Turk and through two experiments with a UR5 industrial robot with 12 able-bodied subjects. [ABSTRACT FROM AUTHOR]

A smart home needs to be human-centric, where it tries to fulfill human needs given the devices it has. Various works are developed to provide homes with reasoning and planning capability to fulfill goals, but most do not support complex sequence of plans or require significant manual effort in devising subplans. This is further aggravated by the need to optimize conflicting personal goals. A solution is to solve the planning problem represented as constraint satisfaction problem (CSP). But CSP uses hard constraints and, thus, cannot handle optimization and partial goal fulfillment efficiently. This paper aims to extend this approach to weighted CSP. Knowledge representation to help in generating planning rules is also proposed, as well as methods to improve performances. Case studies show that the system can provide intelligent and complex plans from activities generated from semantic annotations of the devices, as well as optimization to maximize personal constraints’ fulfillment. Note to Practitioners—Smart home should maximize the fulfillment of personal goals that are often conflicting. For example, it should try to fulfill as much as possible the requests made by both the mother and daughter who wants to watch TV but both having different channel preferences. That said, every person has a set of goals or constraints that they hope the smart home can fulfill. Therefore, human-centric system that automates the loosely coupled devices of the smart home to optimize the goals or constraints of individuals in the home is developed. Automated planning is done using converted services extracted from devices, where conversion is done using existing tools and concepts from Web technologies. Weighted constraint satisfaction that provides the declarative approach to cover large problem domain to realize the automated planner with optimization capability is proposed. Details to speed up planning through search space reduction are also given. Real-time case studies are run in a prototype smart home to demonstrate its applicability and intelligence, where every planning is performed under a maximum of 10 s. The vision of this paper is to be able to implement such system in a community, where devices everywhere can cooperate to ensure the well-being of the community. [ABSTRACT FROM AUTHOR]

Autonomous driving technologies can provide greater safety, comfort, and efficiency for future transportation systems. Until now, much of the research effort has been devoted to developing different sensing and control algorithms. However, there has been limited research on how to handle sensor errors efficiently. A simple error in the sensor may lead to an unexpected failure in the whole autonomous driving function. In those cases, the vehicle is then recommended to be sent back to the manufacturer for repair, which costs time and money. This paper introduces an efficient automatic online sensor correction method. The method includes four major functions: sensor error detection, human teaching, vehicle learning, and vehicle self-evaluation. The first function is assumed to be ready and the major contribution of this paper is the human-vehicle teaching and learning framework, which utilizes human-vehicle interaction to collaboratively adjust the parameter in the control model in order to compensate for the errors of the sensors. The self-evaluation function is also briefly introduced. The applications of this method to radar and vision sensors to recover adaptive cruise control and lane keeping functions are introduced in detail. Experimental results acquired from high-fidelity 1/10-scale autonomous driving vehicles illustrate the effectiveness and advantages of the proposed approach. [ABSTRACT FROM AUTHOR]

Measuring similarities among different nodes is important in graph analysis tasks, such as link prediction, and recommendation. Among different similarity measures, SimRank is one of the most popular and promising ones, and has received a lot of research attention. While most current studies focus on single-pair, single-source/top-k, and all-pairs SimRank computation, few of them have studied finding similar pairs given a set of node pairs, which has attractive applications in personalized search and recommendation tasks. In this paper, we present Carmo, an efficient algorithm for retrieving the top-k similarities from an arbitrary set of pairs. In addition, we introduce two types of indexes to boost the efficiency of Carmo: one is hub-based, the other is tree-based. We show the effectiveness and efficiency of our proposed methods by extensive experiments. [ABSTRACT FROM AUTHOR]

Knowledge representation is one of the critical problems in knowledge engineering and artificial intelligence, while knowledge embedding as a knowledge representation methodology indicates entities and relations in knowledge graph as low-dimensional, continuous vectors. In this way, knowledge graph is compatible with numerical machine learning models. Major knowledge embedding methods employ geometric translation to design score function, which is weak-semantic for natural language processing. To overcome this disadvantage, in this paper, we propose our model based on multi-view clustering framework, which could generate semantic representations of knowledge elements (i.e., entities/relations). With our semantic model, we also present an empowered solution to entity retrieval with entity description. Extensive experiments show that our model achieves substantial improvements against baselines on the task of knowledge graph completion, triple classification, entity classification, and entity retrieval. [ABSTRACT FROM AUTHOR]

The goal of early action recognition is to predict action label when the sequence is partially observed. The existing methods treat the early action recognition task as sequential classification problems on different observation ratios of an action sequence. Since these models are trained by differentiating positive category from all negative classes, the diverse information of different negative categories is ignored, which we believe can be collected to help improve the recognition performance. In this paper, we step towards to a new direction by introducing category exclusion to early action recognition. We model the exclusion as a mask operation on the classification probability output of a pre-trained early action recognition classifier. Specifically, we use policy-based reinforcement learning to train an agent. The agent generates a series of binary masks to exclude interfering negative categories during action execution and hence help improve the recognition accuracy. The proposed method is evaluated on three benchmark recognition datasets, NTU-RGBD, First-Person Hand Action, as well as UCF-101. The proposed method enhances the recognition accuracy consistently over all different observation ratios on the three datasets, where the accuracy improvements on the early stages are especially significant. [ABSTRACT FROM AUTHOR]

Optimal resource provisioning is essential for scalable big data analytics. However, it has been difficult to accurately forecast the resource requirements before the actual deployment of these applications as their resource requirements are heavily application and data dependent. This paper identifies the existence of effective memory resource requirements for most of the big data analytic applications running inside JVMs in distributed Spark environments. Provisioning memory less than the effective memory requirement may result in rapid deterioration of the application execution in terms of its total execution time. A machine learning-based prediction model is proposed in this paper to forecast the effective memory requirement of an application given its service level agreement. This model captures the memory consumption behavior of big data applications and the dynamics of memory utilization in a distributed cluster environment. With an accurate prediction of the effective memory requirement, it is shown that up to 60 percent savings of the memory resource is feasible if an execution time penalty of 10 percent is acceptable. The accuracy of the model is evaluated on a physical Spark cluster with 128 cores and 1TB of total memory. The experiment results show that the proposed solution can predict the minimum required memory size for given acceptable delays with high accuracy, even if the behavior of target applications is unknown during the training of the model. [ABSTRACT FROM AUTHOR]

In recent years, visual question answering (VQA) has become topical. The premise of VQA's significance as a benchmark in AI, is that both the image and textual question need to be well understood and mutually grounded in order to infer the correct answer. However, current VQA models perhaps ‘understand’ less than initially hoped, and instead master the easier task of exploiting cues given away in the question and biases in the answer distribution. In this paper we propose the inverse problem of VQA (iVQA). The iVQA task is to generate a question that corresponds to a given image and answer pair. We propose a variational iVQA model that can generate diverse, grammatically correct and content correlated questions that match the given answer. Based on this model, we show that iVQA is an interesting benchmark for visuo-linguistic understanding, and a more challenging alternative to VQA because an iVQA model needs to understand the image better to be successful. As a second contribution, we show how to use iVQA in a novel reinforcement learning framework to diagnose any existing VQA model by way of exposing its belief set: the set of question-answer pairs that the VQA model would predict true for a given image. This provides a completely new window into what VQA models ‘believe’ about images. We show that existing VQA models have more erroneous beliefs than previously thought, revealing their intrinsic weaknesses. Suggestions are then made on how to address these weaknesses going forward. [ABSTRACT FROM AUTHOR]

This paper describes our long journey from the first standardization projects to support a fully automated work flow to three complex automation hardware projects, including learnings about change management within a running fab for all departments. The general principles used to automate a mature fab were motivated by a clear economical approach – cost reduction in manufacturing by automation of simple and recurring logistic steps. This can only be successfully completed by an interdisciplinary team work. But there is no ONE recipe for all mature fabs – the older the fab, the more flexible and cost effective the automation concepts must be. Collaboration between human and machine become a key element in this context. One topic in this field is the development of smarter and cost efficient mobile robotic systems (AGV). Smarter means more intelligent, less cost and more stability in process execution. Since we manufacture IoT elements like Time of Flight and Radar sensor in our fabs – we also developed a new integrated sensor fusion concept combined with artificial intelligence (AI) and machine Learning (ML) methods. [ABSTRACT FROM AUTHOR]

In this paper, we provide a systematical overview and analysis on the joint feature and texture representation framework, which aims to smartly and coherently represent the visual information with the front-end intelligence in the scenario of video big data applications. In particular, we first demonstrate the advantages of the joint compression framework in terms of both reconstruction quality and analysis accuracy. Subsequently, the interactions between visual feature and texture in the compression process are further illustrated. Finally, the future joint coding scheme by incorporating the deep learning features is envisioned, and future challenges toward seamless and unified joint compression are discussed. The joint compression framework, which bridges the gap between visual analysis and signal-level representation, is expected to contribute to a series of applications, such as video surveillance and autonomous driving. [ABSTRACT FROM AUTHOR]

In this paper, a deep mixture of diverse experts algorithm is developed to achieve more efficient learning of a huge (mixture) network for large-scale visual recognition application. First, a two-layer ontology is constructed to assign large numbers of atomic object classes into a set of task groups according to the similarities of their learning complexities, where certain degrees of inter-group task overlapping are allowed to enable sufficient inter-group message passing. Second, one particular base deep CNNs with $M+1$ M + 1 outputs is learned for each task group to recognize its $M$ M atomic object classes and identify one special class of “not-in-group”, where the network structure (numbers of layers and units in each layer) of the well-designed deep CNNs (such as AlexNet, VGG, GoogleNet, ResNet) is directly used to configure such base deep CNNs. For enhancing the separability of the atomic object classes in the same task group, two approaches are developed to learn more discriminative base deep CNNs: (a) our deep multi-task learning algorithm that can effectively exploit the inter-class visual similarities; (b) our two-layer network cascade approach that can improve the accuracy rates for the hard object classes at certain degrees while effectively maintaining the high accuracy rates for the easy ones. Finally, all these complementary base deep CNNs with diverse but overlapped outputs are seamlessly combined to generate a mixture network with larger outputs for recognizing tens of thousands of atomic object classes. Our experimental results have demonstrated that our deep mixture of diverse experts algorithm can achieve very competitive results on large-scale visual recognition. [ABSTRACT FROM AUTHOR]

Automatic modulation recognition (AMR) is an essential and challenging topic in the development of the cognitive radio (CR), and it is a cornerstone of CR adaptive modulation and demodulation capabilities to sense and learn environments and make corresponding adjustments. AMR is essentially a classification problem, and deep learning achieves outstanding performances in various classification tasks. So, this paper proposes a deep learning-based method, combined with two convolutional neural networks (CNNs) trained on different datasets, to achieve higher accuracy AMR. A CNN is trained on samples composed of in-phase and quadrature component signals, otherwise known as in-phase and quadrature samples, to distinguish modulation modes, that are relatively easy to identify. We adopt dropout instead of pooling operation to achieve higher recognition accuracy. A CNN based on constellation diagrams is also designed to recognize modulation modes that are difficult to distinguish in the former CNN, such as 16 quadratic-amplitude modulation (QAM) and 64 QAM, demonstrating the ability to classify QAM signals even in scenarios with a low signal-to-noise ratio. [ABSTRACT FROM AUTHOR]

In this paper, we propose a joint communication, caching and computing strategy for achieving cost efficiency in vehicular networks. In particular, the resource allocation policy is specifically designed by considering the vehicle's mobility and the hard service deadline constraint. An artificial intelligence-based multi-timescale framework is proposed for tackling these challenges. To mitigate the complexity associated with the large action and search space in the sophisticated multi-timescale framework considered, we propose to maximize a carefully constructed mobility-aware reward function using the classic particle swarm optimization scheme at the associated large timescale level, while we employ deep reinforcement learning at the small timescale level of our sophisticated twin-timescale solution. Numerical results are presented to illustrate the theoretical findings and to quantify the performance gains attained. [ABSTRACT FROM AUTHOR]

Image annotation aims to annotate a given image with a variable number of class labels corresponding to diverse visual concepts. In this paper, we address two main issues in large-scale image annotation: 1) how to learn a rich feature representation suitable for predicting a diverse set of visual concepts ranging from object, scene to abstract concept and 2) how to annotate an image with the optimal number of class labels. To address the first issue, we propose a novel multi-scale deep model for extracting rich and discriminative features capable of representing a wide range of visual concepts. Specifically, a novel two-branch deep neural network architecture is proposed, which comprises a very deep main network branch and a companion feature fusion network branch designed for fusing the multi-scale features computed from the main branch. The deep model is also made multi-modal by taking noisy user-provided tags as model input to complement the image input. For tackling the second issue, we introduce a label quantity prediction auxiliary task to the main label prediction task to explicitly estimate the optimal label number for a given image. Extensive experiments are carried out on two large-scale image annotation benchmark datasets, and the results show that our method significantly outperforms the state of the art. [ABSTRACT FROM AUTHOR]

In recent years, more machine learning algorithms have been applied to odor classification. These odor classification algorithms usually assume that the training data sets are static. However, for some odor recognition tasks, new odor classes continually emerge. That is, the odor data sets are dynamically growing while both training samples and number of classes are increasing over time. Motivated by this concern, this paper proposes a deep nearest class mean (DNCM) model based on the deep learning framework and the nearest class mean method. The proposed model not only leverages deep neural network to extract deep features but also able to dynamically integrate new classes over time. In our experiments, the DNCM model was initially trained with 10 classes, then 25 new classes are integrated. Experiment results demonstrate that the proposed model is very efficient for incremental odor classification, especially for new classes with only a small number of training examples. [ABSTRACT FROM AUTHOR]

Dynamic scene classification has been extensively studied in computer vision due to its widespread applications. The key to dynamic scene classification lies in jointly characterizing spatial appearance and temporal dynamics to achieve informative representation, which remains an outstanding task in the literature. In this paper, we propose a unified framework to extract spatial and temporal features for dynamic scene representation. More specifically, we deploy two variants of deep convolutional neural networks to encode spatial appearance and short-term dynamics into short-term deep features (STDF). Based on STDF, we propose using the autoregressive moving average model to extract long-term frequency features (LTFF). By combining STDF and LTFF, we establish the long–short-term feature (LSTF) representations of dynamic scenes. The LSTF characterizes both spatial and temporal patterns of dynamic scenes for comprehensive and information representation that enables more accurate classification. Extensive experiments on three-dynamic scene classification benchmarks have shown that the proposed LSTF achieves high performance and substantially surpasses the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In this paper, we present a novel and practical approach for benchmarking agility. We focus on terrestrial, multilegged locomotion in the field of bio-inspired robotics. We define agility as the ability to perform a set of different but specific tasks executed in a fast and efficient manner. This definition is inspired by the analysis of natural role models, such as dogs and horses as well as robotic systems. An evaluation of existing benchmarks in robotics is done and taken into account in our proposed benchmark. After the general definition, the actual normalized benchmarking values are defined, and measuring methods, as well as an online database for agility score collection and distribution, are presented. To provide a baseline for agile locomotion, various videos of dog-agility competitions were analyzed and agility scores calculated wherever applicable. Finally, validation and implementation of the benchmark are done with different robots directly available to the authors. In conclusion, our benchmark will enable researchers not only to compare existing robots and find out strengths and weaknesses in different design approaches, but also give a tool to define new fitness functions for optimization, learning processes, and future robots developments, intensifying the links between biology and technology even further. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks (CNNs) are widely adopted in intelligent systems with unprecedented accuracy but at the cost of a substantial amount of data movement. Although the emerging processing-in-memory (PIM) architecture seeks to minimize data movement by placing memory near processing elements, memory is still the major bottleneck in the entire system. The selection of hyper-parameters in the training of CNN applications requires over hundreds of kilobytes cache capacity for concurrent processing of convolutions. How to jointly explore the computation capability of the PIM architecture and the highly parallel property of neural networks remains a critical issue. This paper presents Para-Net, that exploits Parallelism for deterministic convolutional neural Networks on the PIM architecture. Para- Net achieves data-level parallelism for convolutions by fully utilizing the on-chip processing engine (PE) in PIM. The objective is to capture the characteristics of neural networks and present a hardware-independent design to jointly optimize the scheduling of both intermediate results and computation tasks. We formulate this data allocation problem as a dynamic programming model and obtain an optimal solution. To demonstrate the viability of the proposed Para-Net, we conduct a set of experiments using a variety of realistic CNN applications. The graph abstractions are obtained from deep learning framework Caffe. Experimental results show that Para-Net can significantly reduce processing time and improve cache efficiency compared to representative schemes. [ABSTRACT FROM AUTHOR]

The tensor completion issues have obtained a great deal of attention in the past few years. However, the data fidelity part minimizes a squared loss function, which may be inappropriate for the case of noisy one-bit observations. In this paper, we alleviate the mentioned difficulty by drawing on the experience of matrix scenarios. Based on the convex relation to $\ell _{1}$ norm of the tensor multi-rank, we propose a novel optimization model trying to recover the underlying tensor in case of one-bit observations. The feasibility of this model is proved by theoretical derivations. Furthermore, an alternating direction method of multipliers based algorithm is designed to find the solution. The numerical experiments demonstrate the effectiveness of our method. [ABSTRACT FROM AUTHOR]

This paper proposes a new approach to multi-object tracking by semantic topic discovery. We dynamically cluster frame-by-frame detections and treat objects as topics, allowing the application of the Dirichlet process mixture model. The tracking problem is cast as a topic-discovery task, where the video sequence is treated analogously to a document. It addresses tracking issues such as object exclusivity constraints as well as tracking management without the need for heuristic thresholds. Variation of object appearance is modeled as the dynamics of word co-occurrence and handled by updating the cluster parameters across the sequence in the dynamical clustering procedure. We develop two kinds of visual representation based on super-pixel and deformable part model and integrate them into the model of automatic topic discovery for tracking rigid and non-rigid objects, respectively. In experiments on public data sets, we demonstrate the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

The field of object detection has made great progress in recent years. Most of these improvements are derived from using a more sophisticated convolutional neural network. However, in the case of humans, the attention mechanism, global structure information, and local details of objects all play an important role for detecting an object. In this paper, we propose a novel fully convolutional network, named as Attention CoupleNet, to incorporate the attention-related information and global and local information of objects to improve the detection performance. Specifically, we first design a cascade attention structure to perceive the global scene of the image and generate class-agnostic attention maps. Then the attention maps are encoded into the network to acquire object-aware features. Next, we propose a unique fully convolutional coupling structure to couple global structure and local parts of the object to further formulate a discriminative feature representation. To fully explore the global and local properties, we also design different coupling strategies and normalization ways to make full use of the complementary advantages between the global and local information. Extensive experiments demonstrate the effectiveness of our approach. We achieve state-of-the-art results on all three challenging data sets, i.e., a mAP of 85.7% on VOC07, 84.3% on VOC12, and 35.4% on COCO. Codes are publicly available at https://github.com/tshizys/CoupleNet. [ABSTRACT FROM AUTHOR]

This paper presents a systematical framework to solve the multiple unmanned aerial vehicles (multi-UAV) cooperative task assignment problem. Based on a combinatorial optimization model, it is solved by a digraph-based method and a novel meta-heuristic optimization method, named modified two-part wolf pack search (MTWPS) algorithm. When the number of UAVs/targets is large, in order to reduce the simulation time, we also present a new solution framework based on an easy-computing objective function. Additionally, the parameter and time-sensitive uncertainties are considered in the extended task assignment problem. For the problem with parameter uncertainty, it is formulated by a robust optimization method and solved by a novel combined algorithm, including the classical interior point method and our MTWPS algorithm. For the problem with time-sensitive uncertainty, it is solved by a practical online hierarchical planning algorithm. Finally, numerical simulations and physical experiments demonstrate that the proposed methods can provide a flyable solution for the UAVs and achieve outstanding performance in comparison with other algorithms. [ABSTRACT FROM AUTHOR]

This paper presents a natural framework for information sharing in cooperative tasks involving humans and robots. In this framework, all information gathered over time by a human–robot team is exchanged and summarized in the form of a fused probability density function (pdf). An approach for an intelligent system to describe its belief pdfs in English expressions is presented. This belief expression generation is achieved through two goodness measures: semantic correctness and information preservation. In order to describe complex, multimodal belief pdfs, a Mixture of Statements (MoS) model is proposed such that optimal expressions can be generated through compositions of multiple statements. The model is further extended to a nonparametric Dirichlet process MoS generation, such that the optimal number of statements required for describing a given pdf is automatically determined. Results based on information loss, human collaborative task performances, and correctness rating scores suggest that the proposed method for generating belief expressions is an effective approach for communicating probabilistic information between robots and humans. [ABSTRACT FROM AUTHOR]