Search

The aim of this systematic review is to determine the current state of the art in the real-time classification of user-generated content from social media. Focus is on the identification of the main characteristics of data used for training and testing, the types of text processing and normalization that are required, the machine learning methods used most commonly, and how these methods compare to one another in terms of classification performance. Relevant studies were selected from subscription-based digital libraries, free-to-access bibliographies, and self-curated repositories and then screened for relevance with key information extracted and structured against the following facets: natural language processing (NLP) methods, data characteristics, classification methods, and evaluation results. A total of 25 studies published between 2014 and 2018 covering 15 types of classification algorithms were included in this review. Support vector machines (SVMs), Bayesian classifiers, and decision trees were the most commonly employed algorithms with recent emergence of neural network approaches. Domain-specific, application programming interface (API)-driven collection is the most prevalent origin of datasets. The reuse of previously published datasets as a means of benchmarking algorithms against other studies is also prevalent. In conclusion, there are consistent approaches taken when normalizing social media data for text mining and traditional text mining techniques are suited to the task of real-time analysis of social media. more...

The quantity and capabilities of smart devices and sensors deployed as part of the Internet of Things (IoT) and accessible via remote microservices is set to rise dramatically as the provision of interactive data streaming increases. This introduces opportunities to rapidly construct new applications by interconnecting these microservices in different workflow configurations. The challenge is to discover the required microservices, including those from trusted partners and the wider community, whilst being able to operate robustly under diverse networking conditions. This paper outlines a workflow approach that provides decentralized discovery and orchestration of verifiably trustable services in support of multi-party operations. The approach is based on adoption of patterns from self-sovereign identity research, notably Verifiable Credentials, to share information amongst peers based on attestations of service descriptions and prior service usage in a privacy preserving and secure manner. This provides a dynamic, trust-based framework for ratifying and evaluating the qualities of different services. Collating these new service descriptions and integrating with existing decentralized workflow research based on vector symbolic architecture (VSA) provides an enhanced semantic search space for efficient and trusted service discovery that is necessary to support a diverse range of emerging edge-computing environments. An architecture for a dynamic decentralized service discovery system, is designed, and described through application to a scenario which uses trusted peers’ reported experiences of an anomaly detection service to determine service selection. more...

Numerous workflow systems span multiple scientific domains and environments, and for the Internet of Things (IoT), Node-RED offers an attractive Web based user interface to execute IoT service-based workflows. However, like most workflow systems, it coordinates the workflow centrally, and cannot run within more transient environments where nodes are mobile. To address this gap, we show how Node-RED workflows can be migrated into a decentralized execution environment for operation on mobile ad-hoc networks, and we demonstrate this by converting a Node-RED based traffic congestion detection workflow to operate in a decentralized environment. The approach uses a Vector Symbolic Architecture (VSA) to dynamically convert Node-Red applications into a compact semantic vector representation that encodes the service interfaces and the workflow in which they are embedded. By extending existing services interfaces, with a simple cognitive layer that can interpret and exchange the vectors, we show how the required services can be dynamically discovered and interconnected into the required workflow in a completely decentralized manner. The resulting system provides a convenient environment where the Node-RED front-end graphical composition tool can be used to orchestrate decentralized workflows. In this paper, we further extend this work by introducing a new dynamic VSA vector compression scheme that compresses vectors for on-the-wire communication, thereby reducing communication bandwidth while maintaining the semantic information content. This algorithm utilizes the holographic properties of the symbolic vectors to perform compression taking into consideration the number of combined vectors along with similarity bounds that determine conflict with other encoded vectors used in the same context. The resulting savings make this approach extremely efficient for discovery in service-based decentralized workflows. more...

Saliency maps are a popular approach to creating post-hoc explanations of image classifier outputs. These methods produce estimates of the relevance of each pixel to the classification output score, which can be displayed as a saliency map that highlights important pixels. Despite a proliferation of such methods, little effort has been made to quantify how good these saliency maps are at capturing the true relevance of the pixels to the classifier output (i.e. their "fidelity"). We therefore investigate existing metrics for evaluating the fidelity of saliency methods (i.e. saliency metrics). We find that there is little consistency in the literature in how such metrics are calculated, and show that such inconsistencies can have a significant effect on the measured fidelity. Further, we apply measures of reliability developed in the psychometric testing literature to assess the consistency of saliency metrics when applied to individual saliency maps. Our results show that saliency metrics can be statistically unreliable and inconsistent, indicating that comparative rankings between saliency methods generated using such metrics can be untrustworthy., Comment: Accepted for publication at the Thirty Fourth AAAI conference on Artificial Intelligence (AAAI-20) more...

Future Multi-Domain Operations (MDO) will require the coordination of hundreds—even thousands—of devices and component services. This will demand the capability to rapidly discover the distributed devices/services and combine them into different workflow configurations, thereby creating the applications necessary to support changing mission needs. Motivated by neuromorphic processing models, in previous work it was shown that this can be achieved by using hyperdimensional symbolic semantic vector representations of the services/devices and workflows. Using a process of vector exchange the required services are dynamically discovered and inter-connected to achieve the required tasks. In network edge environments, the capability to perform these tasks with minimum energy consumption is critical. This paper describes how emerging spiking neural network (SNN) neuromorphic processing devices can be used to perform the required hyperdimensional vector computation (HDC) with significant energy savings compared to what can be achieved using traditional CMOS implementations. more...

Online social networks serve as a convenient way to connect, share, and promote content with others. As a result, these networks can be used with malicious intent, causing disruption and harm to public debate through the sharing of misinformation. However, automatically identifying such content through its use of natural language is a significant challenge compared to our solution which uses less computational resources, language-agnostic and without the need for complex semantic analysis. Consequently alternative and complementary approaches are highly valuable. In this paper, we assess content that has the potential for misinformation and focus on patterns of user association with online social media communities (subreddits) in the popular Reddit social media platform, and generate networks of behaviour capturing user interaction with different subreddits. We examine these networks using both global and local metrics, in particular noting the presence of induced substructures (graphlets) assessing 7 , 876 , 064 posts from 96,634 users. From subreddits identified as having potential for misinformation, we note that the associated networks have strongly defined local features relating to node degree — these are evident both from analysis of dominant graphlets and degree-related global metrics. We find that these local features support high accuracy classification of subreddits that are categorised as having the potential for misinformation. Consequently we observe that induced local substructures of high degree are fundamental metrics for subreddit classification, and support automatic detection capabilities for online misinformation independent from any particular language. more...

Social media has become extremely influential when it comes to policy making in modern societies, especially in the western world, where platforms such as Twitter allow users to follow politicians, thus making citizens more involved in political discussion. In the same vein, politicians use Twitter to express their opinions, debate among others on current topics and promote their political agendas aiming to influence voter behaviour. In this paper, we attempt to analyse tweets of politicians from three European countries and explore the virality of their tweets. Previous studies have shown that tweets conveying negative sentiment are likely to be retweeted more frequently. By utilising state-of-the-art pre-trained language models, we performed sentiment analysis on hundreds of thousands of tweets collected from members of parliament in Greece, Spain and the United Kingdom, including devolved administrations. We achieved this by systematically exploring and analysing the differences between influential and less popular tweets. Our analysis indicates that politicians' negatively charged tweets spread more widely, especially in more recent times, and highlights interesting differences between political parties as well as between politicians and the general population., Accepted at "Online Social Networks and Media, Volume 33"; for code and data used see https://github.com/cardiffnlp/politics-and-virality-twitter more...

The ubiquity of mobile, wearable, and IoT devices enhances humans with a network of environmental sensors. These devices capture raw, time-series measurements of scalar physical phenomena. To transform the data into human-digestible representations, deep learning methods have enabled high-level interpretations of the opaque raw sensory data. However, interfacing models with humans requires flexibility to support the vast database of human inquiries about sensor data. Deep learning models are usually trained to perform fixed tasks, limiting the inference outputs to a predefined set of high-level labels. To enable flexible inference, we introduce DeepSQA, a generalized Sensory Question Answering (SQA) framework that aims to enable natural language questions about raw sensory data in distributed and heterogeneous IoT networks. Given a sensory data context and a natural language question about the data, the task is to provide an accurate natural language answer. In addition to the DeepSQA, we create SQA-Gen, a software framework for generating SQA datasets using labeled source sensory data, and also generate OppQA with SQA-Gen for benchmarking different SQA models. We evaluate DeepSQA across several state-of-the-art QA models and lay the foundation and challenges for future SQA research. We further provide open-source implementations of the framework, the dataset generation tool, and access to the generated dataset, to help facilitate research on the SQA problem. more...

We provide a description of the DAIS-ITA experimentation environment with accompanying scenario. These were created to aid our research by providing a common platform upon which different projects can build. The purpose is to speed up the innovation pipeline by more closely linking research with impact. The experimentation environment is based on a 3d virtual world. We provide a variety of options for displaying, moving and interacting with objects within the world. Our focus for these interactions is interactivity, either manual or programmatic, such that DAIS research, experiments, simulations and demonstrations can be accurately modelled. There is an interface, based upon the MQTT messaging protocol, that allows input to and output from the world. The world we have created is mapped to the same location as the Anglova Scenario. We are able to use and extend the Anglova Scenario and it’s experimentation data within the world for conducting DAIS research. more...

Recent years have seen significant advances in artificial intelligence (AI) and machine learning (ML) technologies applicable to coalition situational understanding (CSU). However, state-of-the-art ML techniques based on deep neural networks require large volumes of training data; unfortunately, representative training examples of situations of interest in CSU are usually sparse. Moreover, to be useful, ML-based analytic services must be capable of explaining their outputs. We describe an integrated CSU architecture that combines neural networks with symbolic learning and reasoning to address the problem of sparse training data. We also demonstrate how explainability can be achieved for deep neural networks operating on multimodal sensor feeds. The work focuses on real-time decision making settings at the tactical edge, with both the symbolic and neural network parts of the system --- including the explainabilty approaches --- able to deal with temporal features. more...

Social media is often used by individuals and organisations as a platform to spread misinformation. With the recent coronavirus pandemic we have seen a surge of misinformation on Twitter, posing a danger to public health. In this paper, we compile a large COVID-19 misinformation-related Twitter corpus and perform an analysis to discover patterns with respect to vocabulary usage. Among others, our analysis reveals that the variety of topics and vocabulary usage are considerably more limited and negative in tweets related to misinformation than in randomly extracted tweets. In addition to our qualitative analysis, our experimental results show that a simple linear model based only on lexical features is effective in identifying misinformation-related tweets (with accuracy over 80%), providing evidence to the fact that the vocabulary used in misinformation largely differs from generic tweets. © 2021 Association for Computational Linguistics. more...

Adopting shared data resources requires scientists to place trust in the originators of the data. When shared data is later used in the development of artificial intelligence (AI) systems or machine learning (ML) models, the trust lineage extends to the users of the system, typically practitioners in fields such as healthcare and finance. Practitioners rely on AI developers to have used relevant, trustworthy data, but may have limited insight and recourse. This paper introduces a software architecture and implementation of a system based on design patterns from the field of self-sovereign identity. Scientists can issue signed credentials attesting to qualities of their data resources. Data contributions to ML models are recorded in a bill of materials (BOM), which is stored with the model as a verifiable credential. The BOM provides a traceable record of the supply chain for an AI system, which facilitates on-going scrutiny of the qualities of the contributing components. The verified BOM, and its linkage to certified data qualities, is used in the AI Scrutineer, a web-based tool designed to offer practitioners insight into ML model constituents and highlight any problems with adopted datasets, should they be found to have biased data or be otherwise discredited., Comment: This is the submitted, pre-peer reviewed version of this paper more...

Trolling, echo chambers and general suspicious behaviour online are a serious cause of concern due to their potential disruptive effects beyond social media. This motivates a better understanding of the characteristics of disruptive behaviour on the internet and methods of detection. In this work we focus on Reddit which provides a rich social media platform for community focused interactions. Using network representations of user activity alongside temporal statistics and other features we assess the behaviour of a sample of potentially disruptive users, based on their assigned comment karma (an aggregate of a user's comment up-votes), relative to the wider population. We explore how these signals contribute to the accurate prediction of disruptive users, and note that this is achieved without requiring any semantic analysis. Our results show that it is possible to detect signs of disruptive behaviour with good accuracy using limited inputs that are primarily based on the reply patterns that users generate. This is of potential value for large-scale detection problems and operation across different languages. more...

Increased adoption of artificial intelligence (AI) systems into scientific workflows will result in an increasing technical debt as the distance between the data scientists and engineers who develop AI system components and scientists, researchers and other users grows. This could quickly become problematic, particularly where guidance or regulations change and once-acceptable best practice becomes outdated, or where data sources are later discredited as biased or inaccurate. This paper presents a novel method for deriving a quantifiable metric capable of ranking the overall transparency of the process pipelines used to generate AI systems, such that users, auditors and other stakeholders can gain confidence that they will be able to validate and trust the data sources and contributors in the AI systems that they rely on. The methodology for calculating the metric, and the type of criteria that could be used to make judgements on the visibility of contributions to systems are evaluated through models published at ModelHub and PyTorch Hub, popular archives for sharing science resources, and is found to be helpful in driving consideration of the contributions made to generating AI systems and approaches towards effective documentation and improving transparency in machine learning assets shared within scientific communities., Comment: This is the pre-peer reviewed version of the following article: Barclay I, Taylor H, Preece A, Taylor I, Verma D, de Mel G. A framework for fostering transparency in shared artificial intelligence models by increasing visibility of contributions. Concurrency Computat Pract Exper. 2020;e6129. arXiv admin note: substantial text overlap with arXiv:1907.03483 more...

Fifth generation (5G) mobile networks will revolutionize edge-based computing by providing fast and reliable network capabilities to remote sensors, devices and microservices. This heralds new opportunities for researchers, allowing remote instrumentation and analytic capabilities to be as accessible as local resources. The increased availability of remote data and services presents new opportunities for collaboration, yet introduces challenges for workflow orchestration, which will need to adapt to consider an increased choice of available services, including those from trusted partners and the wider community. In this paper we outline a workflow approach that provides decentralized discovery and orchestration of verifiably trustable services in support of multi-party operations. We base this work on the adoption of standardised data models and protocols emerging from hypermedia research, which has demonstrated success in using combinations of Linked Data, Web of Things (WoT) and semantic technologies to provide mechanisms for autonomous goal-directed agents to discover, execute and reuse new heterogeneous resources and behaviours in large-scale, dynamic environments. We adopt Verifiable Credentials (VCs) to securely share information amongst peers based on prior service usage in a cryptographically secure and tamperproof way, providing a trust-based framework for ratifying service qualities. Collating these new service description channels and integrating with existing decentralized workflow research based on vector symbolic architecture (VSA) provides an enhanced semantic search space for efficient and trusted service discovery that will be necessary for 5G edge-computing environments. more...

The focus of this paper is on the generation of multimodal explanations for information fusion tasks performed on multimodal data. We propose that separating modal components in saliency map explanations provides users with a better understanding of how convolutional neural networks process multimodal data. We adapt established state-of-the-art explainability techniques to mid-level fusion networks in order to better understand (a) which modality of the input contributes most to a model's decision and (b) which parts of the input data are most relevant to that decision. Our method separates temporal from non-temporal information to allow a user to focus their attention on salient elements of the scene that are changing in multiple modalities. The work is experimentally tested on an activity recognition task using video and audio data. In view of the fact that explanations need to be tailored to the type of user in a User Fusion context, we focus on meeting explanation requirements for system creators and operators respectively. more...

Coalition situational understanding (CSU) is fundamental to support decision-making and autonomy in multi-domain operations involving multiple allied partners. Our work aims to advance the algorithms and techniques to develop CSU, addressing key scientific challenges in how different levels of representation, reasoning and machine learning (ML) interact with one another to facilitate flow of information and management of uncertainty between coalition agents and services. The very existence of a coalition is contingent on the premise that the whole is greater than the sum of the parts, i.e., the shared model of the environment - acquired using the information learned, combined and inferred from all the agents - is not only more complete but also more robust than local models. Specifically, two aspects of achieving this CSU vision are considered in this paper: (1) integrating learning and reasoning techniques for CSU, addressing the technical challenge of dealing with uncertainly in the more...

Recent rapid progress in machine learning (ML), particularly so‐called ‘deep learning’, has led to a resurgence in interest in explainability of artificial intelligence (AI) systems, reviving an area of research dating back to the 1970s. The aim of this article is to view current issues concerning ML‐based AI systems from the perspective of classical AI, showing that the fundamental problems are far from new, and arguing that elements of that earlier work offer routes to making progress towards explainable AI today. more...

Pre-trained language models provide the foundations for state-of-the-art performance across a wide range of natural language processing tasks, including text classification. However, most classification datasets assume a large amount labeled data, which is commonly not the case in practical settings. In particular, in this paper we compare the performance of a light-weight linear classifier based on word embeddings, i.e., fastText (Joulin et al., 2017), versus a pre-trained language model, i.e., BERT (Devlin et al., 2019), across a wide range of datasets and classification tasks. In general, results show the importance of domain-specific unlabeled data, both in the form of word embeddings or language models. As for the comparison, BERT outperforms all baselines in standard datasets with large training sets. However, in settings with small training datasets a simple method like fastText coupled with domain-specific word embeddings performs equally well or better than BERT, even when pre-trained on domain-specific data. more...

Despite the remarkable success in a broad set of sensing applications, state-of-the-art deep learning techniques struggle with complex reasoning tasks across a distributed set of sensors. Unlike recognizing transient complex activities (e.g., human activities such as walking or running) from a single sensor, detecting more complex events with larger spatial and temporal dependencies across multiple sensors is extremely difficult, e.g., utilizing a hospital's sensor network to detect whether a nurse is following a sanitary protocol as they traverse from patient to patient. Training a more complicated model requires a larger amount of data-which is unrealistic considering complex events rarely happen in nature. Moreover, neural networks struggle with reasoning about serial, aperiodic events separated by large quantities in the spatial-temporal dimensions. We propose Neuroplex, a neural-symbolic framework that learns to perform complex reasoning on raw sensory data with the help of high-level, injected human knowledge. Neuroplex decomposes the entire complex learning space into explicit perception and reasoning layers, i.e., by maintaining neural networks to perform low-level perception tasks and neurally reconstructed reasoning models to perform high-level, explainable reasoning. After training the neurally reconstructed reasoning model using human knowledge, Neuroplex allows effective end-to-end training of perception models with an additional semantic loss using only sparse, high-level annotations. Our experiments and evaluation show that Neuroplex is capable of learning to efficiently and effectively detect complex events-which cannot be handled by state-of-the-art neural network models. During the training, Neuroplex not only reduces data annotation requirements by 100x, but also significantly speeds up the learning process for complex event detection by 4x. more...

Automated negotiation has been used in a variety of distributed settings, such as privacy in the Internet of Things (IoT) devices and power distribution in Smart Grids. The most common protocol under which these agents negotiate is the Alternating Offers Protocol (AOP). Under this protocol, agents cannot express any additional information to each other besides a counter offer. This can lead to unnecessarily long negotiations when, for example, negotiations are impossible, risking to waste bandwidth that is a precious resource at the edge of the network. While alternative protocols exist which alleviate this problem, these solutions are too complex for low power devices, such as IoT sensors operating at the edge of the network. To improve this bottleneck, we introduce an extension to AOP called Alternating Constrained Offers Protocol (ACOP), in which agents can also express constraints to each other. This allows agents to both search the possibility space more efficiently and recognise impossible situations sooner. We empirically show that agents using ACOP can significantly reduce the number of messages a negotiation takes, independently of the strategy agents choose. In particular, we show our method significantly reduces the number of messages when an agreement is not possible. Furthermore, when an agreement is possible it reaches this agreement sooner with no negative effect on the utility. more...

Communicating information between entities requires there to be a common understanding of the language and the concepts being exchanged. Where information has to be rapidly conveyed, or where the communications are limited, the use of specialized language or symbols is often used but this requires the various parties to know the precise meaning of the terminology (i.e. the parties can correctly interpret the language or symbols used). In this paper we propose a new potential approach for information exchange that makes use of binary symbolic vectors to semantically represent the information to be exchanged. We show how such vector representations can be used to semantically convey large amounts information between coalition partners in a compact representation that can be interpreted at different levels of semantic abstraction. In this paper we illustrate the concept by showing how a complex text (in this case Shakespeare's play Hamlet) can be represented using symbolic vectors. We discuss how such representations can be efficiently communicated between coalition partners and how this can significantly reduce the communications requirements. We develop the concept to illustrate how symbolic vectors could be used to perform command and control functions in distributed decentralized coalition operations. more...

We address the problem of feature importance. Often, when working with classification or regression problems, the results of black-box deep learning techniques are held to scrutiny in an effort to interpret which and to what extent various features affect outcome. We address this issue specifically when the model has a bottleneck which we will be used to infer feature importance. In this paper, we apply this technique to weather data and study which weather features affect traffic most. To this end, we introduce convolutional spatial embedding to convert data with spatial information into spatial images that are suitable for convolutional neural networks. An advantage of our approach is in dealing with input that has highly correlated features, where removing even an important feature will not increase loss. more...

Traditional service-based applications, in fixed networks, are typically constructed and managed centrally and assume stable service endpoints and adequate network connectivity. Constructing and maintaining such applications in dynamic heterogeneous wireless networked environments, where limited bandwidth and transient connectivity are commonplace, presents significant challenges and makes centralized application construction and management impossible. In this demonstration we present an architecture which is capable of providing an adaptable and resilient method for on-demand decentralized construction and management of complex time-critical applications in such environments. The approach uses a Vector Symbolic Architecture (VSA) to compactly represent an application as a single semantic vector that encodes the service interfaces, workflow, and the time-critical constraints required. By extending existing services interfaces, with a simple cognitive layer that can interpret and exchange the vectors, we show how the required services can be dynamically discovered and interconnected in a completely decentralized manner. There are a large number of workflow systems designed to work in various scientific domains, including support for the Internet of Things (IoT). One such workflow system is Node-RED, which is designed to bring workflow-based programming to IoT. The main focus of this demonstration is to show how we can migrate Node-RED workflows into a decentralized execution environment, so that such workflows can run on Edge networks. more...

Situational understanding is impossible without causal reasoning and reasoning under and about uncertainty, i.e. probabilistic reasoning and reasoning about the confidence in the uncertainty assessment. We therefore consider the case of subjective (uncertain) Bayesian networks. In previous work we notice that when observations are out of the ordinary, confidence decreases because the relevant training data, effective instantiations, to determine the probabilities for unobserved variables, on the basis of the observed variables, is significantly smaller than the size of the training data, the total number of instantiations. It is therefore of primary importance for the ultimate goal of situational understanding to be able to efficiently determine the reasoning paths that lead to low confidence whenever and wherever it occurs: this can guide specific data collection exercises to reduce such an uncertainty. We propose three methods to this end, and we evaluate them on the basis of a case-study developed in collaboration with professional intelligence analysts. more...

Based on current capabilities, many Machine Learning techniques are often inscrutable and they can be hard for users to trust because they lack effective means of generating explanations for their outputs. There is much research and development investigating this area, with a wide variety of proposed explanation techniques for AI/ML across a variety of data modalities. In this paper we investigate which modality of explanation to choose for a particular user and task, taking into account relevant contextual information such as the time available to them, their level of skill, what level of access they have to the data and sensors in question, and the device that they are using. Additional environmental factors such as available bandwidth, currently usable sensors and services are also able to be accounted for. The explanation techniques that we are investigating range across transparent and post-hoc mechanisms and form part of a conversation with the user in which the explanation (and therefore human understanding of the AI decision) can be ascertained through dialogue with the system. Our research is exploring generic techniques that can be used to underpin useful explanations in a range of modalities in the context of AI/ML services that operate on multisensor data in a distributed, dynamic, contested and adversarial setting. We define a meta-model for representing this information and through a series of examples show how this approach can be used to support conversational explanation across a range of situations, datasets and modalities. more...

This article provides a case study analysis of social reactions to the murder of Fusilier Lee Rigby in 2013. Informed by empirical data collected by systematic monitoring of social media platforms, the analysis identifies a number of online behaviours with offline effects—labeled the ten “Rs”—that collectively constitute the process of social reaction to the crime. These are defined as: reporting; requesting; responding; recruiting; “risking”; retaliating; rumouring; remembering; reheating; and “resiliencing”. It is argued that the ability to observe these behaviours through the application of qualitative social media analysis has considerable potential. Conceptually, the analysis provides new insight into the complex and chaotic processes of sense-making and meaning attribution that arise in the aftermath of terrorist attacks. It illuminates how patterns of social reaction on social media are nuanced and complicated, with different segments of the public interpreting the same developments very differently. In addition, the findings and the conceptual framework outlined have implications for policy and practice development in terms of establishing a more effective and evidence-based approach to the consequence management of “post-event” conflict dynamics and social reactions. more...

A key problem in managing intelligence, surveillance and reconnaissance (ISR) operations in a coalition context is assigning available sensing assets—of which there are increasingly many—to mission tasks. High demands for information and relative scarcity of available assets implies that assignments must be made taking into account all possible ways of achieving an ISR task by different kinds of sensing. Moreover, the dynamic nature of most ISR situations means that asset assignment must be done in a highly agile manner. The problem is exacerbated in a coalition context because it is harder for users to have an overview of all suitable assets across multiple coalition partners. In this chapter, we describe a knowledge-driven approach to ISR asset assignment using ontologies, allocation algorithms, and a service-oriented architecture and we analyse the approach of using a representation based on Controlled English (CE) to improve the interface and human-in-the-loop aspects of the sensor assignment. An illustration of the use of the system from a mobile device is presented. more...

Deep learning models typically make inferences over transient features of the latent space, i.e., they learn data representations to make decisions based on the current state of the inputs over short periods of time. Such models would struggle with state-based events, or complex events, that are composed of simple events with complex spatial and temporal dependencies. In this paper, we propose DeepCEP, a framework that integrates the concepts of deep learning models with complex event processing engines to make inferences across distributed, multimodal information streams with complex spatial and temporal dependencies. DeepCEP utilizes deep learning to detect primitive events. A user can define a complex event to be detected as a particular sequence or pattern of primitive events as well as any other logical predicates that constrain the definition of such an event. The integration of human logic not only increases robustness and interpretability, but also greatly reduces the amount of training data required. Further, we demonstrate how the uncertainty of a model can be propagated throughout the complex event detection pipeline. Finally, we enumerate the future directions of research enabled by DeepCEP. In particular, we detail how an end-to-end training model for complex event processing with deep learning may be realized. more...

There are a large number of workflow systems designed to work in various scientific domains, including support for the Internet of Things (IoT). One such workflow system is Node-RED, which is designed to bring workflow-based programming to IoT. However, the majority of scientific workflow systems, and specifically systems like Node-RED, are designed to operate in a fixed networked environment, which rely on a central point of coordination in order to manage the workflow. The main focus of the work described in this paper is to investigate means whereby we can migrate Node-RED workflows into a decentralized execution environment, so that such workflows can run on Edge networks, where nodes are extremely transient in nature. In this work, we demonstrate the feasibility of such an approach by showing how we can migrate a Node-RED based traffic congestion detection workflow into a decentralized environment. The detection algorithm is implemented as a set of Web services within Node-RED and we have architected and implemented a system that proxies the centralized Node-RED services using cognitively-aware wrapper services, designed to operate in a decentralized environment. Our cognitive services use a Vector Symbolic Architecture to semantically represent service descriptions and workflows in a way that can be unraveled on the fly without any central point of control. The VSA-based system is capable of parsing Node-RED workflows and migrating them to a decentralized environment for execution; providing a way to use Node-RED as a front-end graphical composition tool for decentralized workflows. more...

In this paper we provide a critical analysis with met- rics that will inform guidelines for designing distributed systems for Collective Situational Understanding (CSU). CSU requires both collective insight—i.e., accurate and deep understanding of a situation derived from uncertain and often sparse data and collective foresight—i.e., the ability to predict what will happen in the future. When it comes to complex scenarios, the need for a distributed CSU naturally emerges, as a single monolithic approach not only is unfeasible: it is also undesirable. We therefore propose a principled, critical analysis of AI techniques that can support specific tasks for CSU to derive guidelines for designing distributed systems for CSU. more...

Many important decisions historically made by humans are now being made by algorithms - often learnt from data - whose accountability measures and legal standards are far from satisfactory. While model transparency is important, it is neither necessary nor sufficient. Accountability is arguably more important. However, accountability needs to carefully take into consideration the weaknesses of the original data, as well as the weaknesses of the model itself: indeed, robust datasets enable model robustness, and vice versa. In this paper we will focus on unfair datasets, as an example of the weaknesses in datasets. Fairness directly involves privacy problems, since learning without fairness can emphasize certain features or directions that generate private information leakage. For instance, a model may inadvertently reveal a persons age if age is a discriminating feature in a models decision making. Moreover, we will investigate the robustness of model in presence of adversarial activities. Indeed, we should strengthen our models by estimating what an adversary will do based on continuous dynamic learning, mindful of concealment and deception, and with a clear, explainable, insightful summary for the final decision makers. In this paper we will discuss how models based on unfair datasets can hardly be robust; and datasets used by weak models can hardly be fair. more...

We introduce the Sentinel platform that supports semantic enrichment of streamed social media data for the purposes of situational understanding. The platform is the result of a codesign effort between computing and social scientists, iteratively developed through a series of pilot studies. The platform is founded upon a knowledge-based approach, in which input streams ( channels ) are characterized by spatial and terminological parameters, collected media is preprocessed to identify significant terms ( signals ), and data are tagged ( framed ) in relation to an ontology. Interpretation of processed media is framed in terms of the 5W framework (who, what, when, where, and why). The platform is designed to be open to the incorporation of new processing modules, building on the knowledge-based elements (channels, signals, and framing ontology) and accessible via a set of user-facing apps. We present the conceptual architecture for the platform, discuss the design and implementation challenges of the underlying stream-processing system, and present a number of apps developed in the context of the pilot studies, highlighting the strengths and importance of the codesign approach and indicating promising areas for future research. more...

This article is based upon a case study of the 2013 murder of Fusilier Lee Rigby in Woolwich, London. It shows how analysis of open source communications data collected via social media platforms can illuminate the inter- and intra-community conflict dynamics arising in the aftermath of such events. Framed by Collins’ recent theoretical work on the escalatory and de-escalatory forces in conflict situations, the empirical analysis brings to the fore some new insights about the ‘arc of conflict’. These frame a conceptual accent upon the interactive sequences of mobilization and counter-mobilization occurring in the moves towards group-based conflicts, and the importance of understanding the multi-polar nature of these involvements. more...

Due to copyright restrictions, the access to the full text of this article is only available via subscription. In this paper we present a methodology to exploit human-machine coalitions for situational understanding. Situational understanding refers to the ability to relate relevant information and form logical conclusions, as well as identify gaps in information. This process for comprehension of the meaning information requires the ability to reason inductively, for which we will exploit the machines’ ability to ‘learn’ from data. However, important phenomena are often rare in occurrence with high degrees of uncertainty, thus severely limiting the availability of instance data for training, and hence the applicability of many machine learning approaches. Therefore, we present the benefits of Subjective Bayesian Networks—i.e., Bayesian Networks with imprecise probabilities—for situational understanding, and the role of conversational interfaces for supporting decision makers in the evolution of situational understanding. Army Research Laboratory ; Ministry of Defence more...

—Controlled Natural Language (CNL) has great potential to support human-machine interaction (HMI) because it provides an information representation that is both human readable and machine processable.We investigated the effectiveness of a CNL-based conversational interface for HMI in a behavioural experiment called Simple Human Experiment Regarding Locally Observed Collective Knowledge (SHERLOCK). In SHERLOCK, individuals acted in groups to discover and report information to the machine using natural language (NL), which the machine then processed into CNL. The machine fused responses from different users to form a common operating picture, a dashboard showing the level of agreement for distinct information. To obtain information to add to this dashboard, users explored the real world in a simulated crowd-sourced sensing scenario. This scenario represented a simplified, controlled analogue for tactical intelligence (i.e., direct intelligence of the environment), which is key for rapidly planning military, law enforcement, and emergency operations. Overall, despite close to zero training, 74% of the users inputted NL that was machine interpretable and addressed the assigned tasks. An experimental manipulation aimed to increase user-machine interaction, however, did not improve performance as hypothesised. Nevertheless, results indicate the conversational interface may be effective in assisting humans with collection and fusion of information in a crowd-sourcing context. more...

Situational understanding (SU) requires a combination\ud of insight — the ability to accurately perceive an existing\ud situation — and foresight — the ability to anticipate how\ud an existing situation may develop in the future. SU involves\ud information fusion as well as model representation and inference.\ud Commonly, heterogenous data sources must be exploited in the\ud fusion process: often including both hard and soft data products.\ud In a coalition context, data and processing resources will also be\ud distributed and subjected to restrictions on information sharing.\ud It will often be necessary for a human to be in the loop in SU\ud processes, to provide key input and guidance, and to interpret\ud outputs in a way that necessitates a degree of transparency\ud in the processing: systems cannot be “black boxes”. In this\ud paper, we characterize the Coalition Situational Understanding\ud (CSU) problem in terms of fusion, temporal, distributed, and\ud human requirements. There is currently significant interest in\ud deep learning (DL) approaches for processing both hard and\ud soft data. We analyze the state-of-the-art in DL in relation to\ud these requirements for CSU, and identify areas where there is\ud currently considerable promise, and key gaps. more...

Deep neural networks have achieved near-human accuracy levels in various types of classification and prediction tasks including images, text, speech, and video data. However, the networks continue to be treated mostly as black-box function approximators, mapping a given input to a classification output. The next step in this human-machine evolutionary process — incorporating these networks into mission critical processes such as medical diagnosis, planning and control — requires a level of trust association with the machine output. Typically, statistical metrics are used to quantify the uncertainty of an output. However, the notion of trust also depends on the visibility that a human has into the working of the machine. In other words, the neural network should provide human-understandable justifications for its output leading to insights about the inner workings. We call such models as interpretable deep networks. Interpretability is not a monolithic notion. In fact, the subjectivity of an interpretation, due to different levels of human understanding, implies that there must be a multitude of dimensions that together constitute interpretability. In addition, the interpretation itself can be provided either in terms of the low-level network parameters, or in terms of input features used by the model. In this paper, we outline some of the dimensions that are useful for model interpretability, and categorize prior work along those dimensions. In the process, we perform a gap analysis of what needs to be done to improve model interpretability. more...

Our particular research in the Distributed Analytics\ud and Information Science International Technology Alliance\ud (DAIS ITA) is focused on ”Anticipatory Situational Understanding\ud for Coalitions”.\ud This paper takes the concrete example of detecting and\ud predicting traffic congestion in the UK road transport network\ud from existing generic sensing sources, such as real-time CCTV\ud imagery and video, which are publicly available for this purpose.\ud This scenario has been chosen carefully as we believe that in\ud a typical city, all data relevant to transport network congestion\ud information is not generally available from a single unified source,\ud and that different organizations in the city (e.g. the weather office,\ud the police force, the general public, etc.) have their own different\ud sensors which can provide information potentially relevant to\ud the traffic congestion problem. In this paper we are looking at\ud the problem of (a) identifying congestion using cameras that,\ud for example, the police department may have access to, and (b)\ud fusing that with other data from other agencies in order to (c)\ud augment any base data provided by the official transportation\ud department feeds. By taking this coalition approach this requires\ud using standard cameras to do different supplementary tasks like\ud car counting, and in this paper we examine how well those tasks\ud can be done with RNN/CNN, and other distributed machine\ud learning processes.\ud In this paper we provide details of an initial four-layer\ud architecture and potential tooling to enable rapid formation of\ud human/machine hybrid teams in this setting, with a focus on\ud opportunistic and distributed processing of the data at the edge\ud of the network. In future work we plan to integrate additional\ud data-sources to further augment the core imagery data. more...

Over the last 70 years there has been a major shift in the threats to global peace. While the 1950's and 1960's were characterised by the cold war and the arms race, many security threats are now characterised by group behaviours that are disruptive, subversive or extreme. In many cases such groups are loosely and chaotically organised, but their ideals are sociologically and psychologically embedded in group members to the extent that the group represents a major threat. As a result, insights into how human groups form, emerge and change are critical, but surprisingly limited insights into the mutability of human groups exist. In this paper we argue that important clues to understand the mutability of groups come from examining the evolutionary origins of human behaviour. In particular, groups have been instrumental in human evolution, used as a basis to derive survival advantage, leaving all humans with a basic disposition to navigate the world through social networking and managing their presence in a group. From this analysis we present five critical features of social groups that govern mutability, relating to social norms, individual standing, status rivalry, ingroup bias and cooperation. We argue that understanding how these five dimensions interact and evolve can provide new insights into group mutation and evolution. Importantly, these features lend themselves to digital modeling. Therefore computational simulation can support generative exploration of groups and the discovery of latent factors, relevant to both internal group and external group modelling. Finally we consider the role of online social media in relation to understanding the mutability of groups. This can play an active role in supporting collective behaviour, and analysis of social media in the context of the five dimensions of group mutability provides a fresh basis to interpret the forces affecting groups. more...

In domains such as emergency response, environmental monitoring, policing and security, sensor and information networks are deployed to assist human users across multiple agencies to conduct missions at or near the 'front line'. These domains present challenging problems in terms of human-machine collaboration: human users need to task the network to help them achieve mission objectives, while humans (sometimes the same individuals) are also sources of mission-critical information. We propose a natural language-based conversational approach to supporting humanmachine working in mission-oriented sensor networks. We present a model for human-machine and machine-machine interactions in a realistic mission context, and evaluate the model using an existing surveillance mission scenario. The model supports the flow of conversations from full natural language to a form of Controlled Natural Language (CNL) amenable to machine processing and automated reasoning, including high-level information fusion tasks. We introduce a mechanism for presenting the gist of verbose CNL expressions in a more convenient form for human users. We show how the conversational interactions supported by the model include requests for expansions and explanations of machine-processed information. more...

We investigate automation bias for confirming\ud erroneous information with a conversational interface.\ud Participants in our studies used a conversational interface to\ud report information in a simulated intelligence, surveillance, and\ud reconnaissance (ISR) task. In the task, for flexibility and ease of\ud use, participants reported information to the conversational\ud agent in natural language. Then, the conversational agent\ud interpreted the user’s reports in a human- and machine-readable\ud language. Next, participants could accept or reject the agent’s\ud interpretation. Misparses occur when the agent incorrectly\ud interprets the report and the user erroneously accepts it. We\ud hypothesize that the misparses naturally occur in the experiment\ud due to automation bias and complacency because the agent\ud interpretation was generally correct (92%). These errors indicate\ud some users were unable to maintain situation awareness using\ud the conversational interface. Our results illustrate concerns for\ud implementing a flexible conversational interface in safety critical\ud environments (e.g., military, emergency operations). more...

In the wake of rapid maturing of Internet of Things (IoT) approaches and technologies in the commercial sector,\ud the IoT is increasingly seen as a key ‘disruptive’ technology in military environments. Future operational environments\ud are expected to be characterized by a lower proportion of human participants and a higher proportion of\ud autonomous and semi-autonomous devices. This view is reflected in both US ‘third offset’ and UK ‘information\ud age’ thinking and is likely to have a profound effect on how multinational coalition operations are conducted\ud in the future. Much of the initial consideration of IoT adoption in the military domain has rightly focused\ud on security concerns, reflecting similar cautions in the early era of electronic commerce. As IoT approaches\ud mature, this initial technical focus is likely to shift to considerations of interactivity and policy. In this paper,\ud rather than considering the broader range of IoT applications in the military context, we focus on roles for IoT\ud concepts and devices in future intelligence, surveillance and reconnaissance (ISR) tasks, drawing on experience\ud in sensor-mission resourcing and human-computer collaboration (HCC) for ISR. We highlight the importance of\ud low training overheads in the adoption of IoT approaches, and the need to balance proactivity and interactivity\ud (push vs pull modes). As with sensing systems over the last decade, we emphasize that, to be valuable in ISR\ud tasks, IoT devices will need a degree of mission-awareness in addition to an ability to self-manage their limited\ud resources (power, memory, bandwidth, computation, etc). In coalition operations, the management and potential\ud sharing of IoT devices and systems among partners (e.g., in cross-coalition tactical-edge ISR teams) becomes a\ud key issue due heterogeneous factors such as language, policy, procedure and doctrine. Finally, we briefly outline a\ud platform that we have developed in order to experiment with human-IoT teaming on ISR tasks, in both physical\ud and virtual settings. more...

To support dynamic communities of interests in coalition operations, new architectures for efficient sharing of ISR assets are needed. The use of blockchain technology in wired business environments, such as digital currency systems, offers an interesting solution by creating a way to maintain a distributed shared ledger without requiring a single trusted authority. In this paper, we discuss how a blockchain-based system can be modified to provide a solution for dynamic asset sharing amongst coalition members, enabling the creation of a logically centralized asset management system by a seamless policy-compliant federation of different coalition systems. We discuss the use of blockchain for three different types of assets in a coalition context, showing how blockchain can offer a suitable solution for sharing assets in those environments. We also discuss the limitations in the current implementations of blockchain which need to be overcome for the technology to become more effective in a decentralized tactical edge environment. more...