Search

This is Part II of the two-part comprehensive survey devoted to a computing framework most commonly known under the names Hyperdimensional Computing and Vector Symbolic Architectures (HDC/VSA). Both names refer to a family of computational models that use high-dimensional distributed representations and rely on the algebraic properties of their key operations to incorporate the advantages of structured symbolic representations and vector distributed representations. Holographic Reduced Representations [321, 326] is an influential HDC/VSA model that is well known in the machine learning domain and often used to refer to the whole family. However, for the sake of consistency, we use HDC/VSA to refer to the field. Part I of this survey [222] covered foundational aspects of the field, such as the historical context leading to the development of HDC/VSA, key elements of any HDC/VSA model, known HDC/VSA models, and the transformation of input data of various types into high-dimensional vectors suitable for HDC/VSA. This second part surveys existing applications, the role of HDC/VSA in cognitive computing and architectures, as well as directions for future work. Most of the applications lie within the Machine Learning/Artificial Intelligence domain; however, we also cover other applications to provide a complete picture. The survey is written to be useful for both newcomers and practitioners. [ABSTRACT FROM AUTHOR]

We investigate the task of retrieving information from compositional distributed representations formed by hyperdimensional computing/vector symbolic architectures and present novel techniques that achieve new information rate bounds. First, we provide an overview of the decoding techniques that can be used to approach the retrieval task. The techniques are categorized into four groups. We then evaluate the considered techniques in several settings that involve, for example, inclusion of external noise and storage elements with reduced precision. In particular, we find that the decoding techniques from the sparse coding and compressed sensing literature (rarely used for hyperdimensional computing/vector symbolic architectures) are also well suited for decoding information from the compositional distributed representations. Combining these decoding techniques with interference cancellation ideas from communications improves previously reported bounds (Hersche et al., 2021) of the information rate of the distributed representations from 1.20 to 1.40 bits per dimension for smaller codebooks and from 0.60 to 1.26 bits per dimension for larger codebooks. [ABSTRACT FROM AUTHOR]

This two-part comprehensive survey is devoted to a computing framework most commonly known under the names Hyperdimensional Computing and Vector Symbolic Architectures (HDC/VSA). Both names refer to a family of computational models that use high-dimensional distributed representations and rely on the algebraic properties of their key operations to incorporate the advantages of structured symbolic representations and distributed vector representations. Notable models in the HDC/VSA family are Tensor Product Representations, Holographic Reduced Representations, Multiply-Add-Permute, Binary Spatter Codes, and Sparse Binary Distributed Representations but there are other models too. HDC/VSA is a highly interdisciplinary field with connections to computer science, electrical engineering, artificial intelligence, mathematics, and cognitive science. This fact makes it challenging to create a thorough overview of the field. However, due to a surge of new researchers joining the field in recent years, the necessity for a comprehensive survey of the field has become extremely important. Therefore, amongst other aspects of the field, this Part I surveys important aspects such as: known computational models of HDC/VSA and transformations of various input data types to high-dimensional distributed representations. Part II of this survey [84] is devoted to applications, cognitive computing and architectures, as well as directions for future work. The survey is written to be useful for both newcomers and practitioners. [ABSTRACT FROM AUTHOR]

Operations on high-dimensional, fixed-width vectors can be used to distribute information from several vectors over a single vector of the same width. For example, a set of key-value pairs can be encoded into a single vector with multiplication and addition of the corresponding key and value vectors: the keys are bound to their values with component-wise multiplication, and the key-value pairs are combined into a single superposition vector with component-wise addition. The superposition vector is, thus, a memory which can then be queried for the value of any of the keys, but the result of the query is approximate. The exact vector is retrieved from a codebook (a.k.a. itemmemory), which contains vectors defined in the system. To perform-these operations, the item memory vectors and the superposition vector must be the same width. Increasing the capacity of the memory requires increasing the width of the superposition and item memory vectors. In this article, we demonstrate that in a regime where many (e.g., 1,000 or more) key-value pairs are stored, an associative memory which maps key vectors to value vectors requires less memory and less computing to obtain the same reliability of storage as a superposition vector. These advantages are obtained because the number of storage locations in an associate memory can be increased without increasing the width of the vectors in the item memory. An associative memory would not replace a superposition vector as a medium of storage, but could augment it, because data recalled from an associative memory could be used in algorithms that use a superposition vector. This would be analogous to how human working memory (which stores about seven items) uses information recalled from long-term memory (which is much larger than the working memory). We demonstrate the advantages of an associative memory experimentally using the storage of large finite-state automata, which could model the storage and recall of state-dependent behavior by brains. [ABSTRACT FROM AUTHOR]

This paper presents the use of automated machine learning for solving a practical problem of a real-life Smart Waste Management system. In particular, the focus of the paper is on the problem of detection (i.e., binary classification) of emptying of a recycling container using sensor measurements. Numerous data-driven methods for solving the problem are investigated in a realistic setting where most of the events are not actual emptying. The investigated methods include the existing manually engineered model and its modification as well as conventional machines learning algorithms. The use of machine learning allows improving the classification accuracy and recall of the existing manually engineered model from $86.8\%$ and $47.9\%$ to $99.1\%$ and $98.2\%$ , respectively, when using the best performing solution. This solution uses a Random Forest classifier on a set of features based on the filling level at different given time spans. Finally, compared to the baseline existing manually engineered model, the best performing solution also improves the quality of forecasts for emptying time of recycling containers. [ABSTRACT FROM AUTHOR]

A data-mining-based method for real-time detection of radio jamming denial-of-service attacks in the IEEE 802.11p vehicle-to-vehicle (V2V) communications is proposed. The method aims at understanding the reasons for losses of periodic cooperative awareness messages (CAMs) exchanged by vehicles in a platoon. Detection relies on a knowledge of the IEEE 802.11p protocol rules as well as on the historical observation of events in the V2V channel. In comparison with the state-of-the-art method, the proposed method allows operating under the realistic assumption of random jitter accompanying every CAM transmission. The method is evaluated for two jamming models: random and ON–OFF. [ABSTRACT FROM AUTHOR]

Inherent unreliability of wireless communications may have crucial consequences when safety-critical C-ITS applications enabled by VANETs are concerned. Although natural sources of packet losses in VANETs such as network traffic congestion are handled by decentralized congestion control (DCC), losses caused by malicious interference need to be controlled too. For example, jamming DoS attacks on CAMs may endanger vehicular safety, and first and foremost are to be detected in real time. Our first goal is to discuss key literature on jamming modeling in VANETs and revisit some existing detection methods. Our second goal is to present and evaluate our own recent results on how to address the real-time jamming detection problem in V2X safety-critical scenarios with the use of AI. We conclude that our hybrid jamming detector, which combines statistical network traffic analysis with data mining methods, allows the achievement of acceptable performance even when random jitter accompanies the generation of CAMs, which complicates the analysis of the reasons for their losses in VANETs. The use case of the study is a challenging platooning C-ITS application, where V2X-enabled vehicles move together at highway speeds with short inter-vehicle gaps. [ABSTRACT FROM AUTHOR]

Objective: Novel minimum-contact vital signs monitoring techniques like textile or capacitive electrocardiogram (ECG) provide new opportunities for health monitoring. These techniques are sensitive to artifacts and require handling of unstable signal quality. Spatio-temporal blind source separation (BSS) is capable of processing suchlike multichannel signals. However, BSS's permutation indeterminacy requires the selection of the cardiac signal (i.e., the component resembling the electric cardiac activity) after its separation from artifacts. This study evaluates different concepts for solving permutation indeterminacy. Methods: Novel automated component selection routines based on heartbeat detections are compared with standard concepts, as using higher order moments or frequency-domain features, for solving permutation indeterminacy in spatio-temporal BSS. BSS was applied to a textile and a capacitive ECG dataset of healthy subjects performing a motion protocol, and to the MIT-BIH Arrhythmia Database. The performance of the subsequent component selection was evaluated by means of the heartbeat detection accuracy (ACC) using an automatically selected single component. Results: The proposed heartbeat-detection-based selection routines significantly outperformed the standard selectors based on Skewness, Kurtosis, and frequency-domain features, especially for datasets containing motion artifacts. For arrhythmia data, beat analysis by sparse coding outperformed simple periodicity tests of the detected heartbeats. Conclusion: Component selection routines based on heartbeat detections are capable of reliably selecting cardiac signals after spatio-temporal BSS in case of severe motion artifacts and arrhythmia. Significance : The availability of robust cardiac component selectors for solving permutation indeterminacy facilitates the usage of spatio-temporal BSS to extract cardiac signals in artifact-sensitive minimum-contact vital signs monitoring techniques. [ABSTRACT FROM AUTHOR]

To accommodate structured approaches of neural computation, we propose a class of recurrent neural networks for indexing and storing sequences of symbols or analog data vectors. These networks with randomized input weights and orthogonal recurrent weights implement coding principles previously described in vector symbolic architectures (VSA) and leverage properties of reservoir computing. In general, the storage in reservoir computing is lossy, and crosstalk noise limits the retrieval accuracy and information capacity. A novel theory to optimize memory performance in such networks is presented and compared with simulation experiments. The theory describes linear readout of analog data and readout with winner-take-all error correction of symbolic data as proposed in VSA models. We find that diverse VSA models from the literature have universal performance properties, which are superior to what previous analyses predicted. Further, we propose novel VSA models with the statistically optimal Wiener filter in the readout that exhibit much higher information capacity, in particular for storing analog data. The theory we present also applies to memory buffers, networks with gradual forgetting, which can operate on infinite data streams without memory overflow. Interestingly, we find that different forgetting mechanisms, such as attenuating recurrent weights or neural nonlinearities, produce very similar behavior if the forgetting time constants are matched. Such models exhibit extensive capacity when their forgetting time constant is optimized for given noise conditions and network size. These results enable the design of new types of VSA models for the online processing of data streams. [ABSTRACT FROM AUTHOR]

This paper proposes a real-time self-adaptive approach for accurate path loss estimation in underground mines or tunnels based on signal strength measurements from heterogeneous radio communication technologies. The proposed model features simplicity of implementation. The methodology is validated in simulations and verified by measurements taken in real environments. The proposed method leverages accuracy of positioning matching the existing approaches while requiring smaller engineering efforts. [ABSTRACT FROM AUTHOR]

The contribution of this article is twofold. First, it presents an encoding approach for seamless bidirectional transitions between localist and distributed representation domains. Second, the approach is demonstrated on the example of using Vector Symbolic Architecture for solving a problem of finding common substrings. The proposed algorithm uses elementary operations on long binary vectors. For the case of two patterns with respective lengths L 1 and L 2 it requires Θ( L 1 + L 2 – 1) operations on binary vectors, which is equal to the suffix trees approach – the fastest algorithm for this problem. The simulation results show that in order to be robustly detected by the proposed approach the length of a common substring should be more than 4% of the longest pattern. [ABSTRACT FROM AUTHOR]

Molecular communication is a relatively new communication paradigm for nanomachines where the communication is realized by utilizing existing biological components found in nature. In recent years researchers have proposed using bacteria to realize molecular communication because the bacteria have the ability to swim and migrate between locations, carry DNA contents (i.e. plasmids) that could be utilized for information storage, and interact and transfer plasmids to other bacteria (one of these processes is known as bacterial conjugation). However, current proposals for bacterial nanonetworks have not considered the internal structures of the nanomachines that can facilitate the use of bacteria as an information carrier. This article presents the types and functionalities of nanomachines that can be utilized in bacterial nanonetworks. A particular focus is placed on the bacterial conjugation and its support for multihop communication between nanomachines. Simulations of the communication process have also been evaluated, to analyze the quantity of bits received as well as the delay performances. Wet lab experiments have also been conducted to validate the bacterial conjugation process. The article also discusses potential applications of bacterial nanonetworks for cancer monitoring and therapy. [ABSTRACT FROM AUTHOR]

Hyperdimensional computing (HD), also known as vector symbolic architectures (VSA), is a framework for computing with distributed representations by exploiting properties of random high-dimensional vector spaces. The commitment of the scientific community to aggregate and disseminate research in this particularly multidisciplinary area has been fundamental for its advancement. Joining these efforts, we present Torchhd, a highperformance open source Python library for HD/VSA. Torchhd seeks to make HD/VSA more accessible and serves as an efficient foundation for further research and application development. The easy-to-use library builds on top of PyTorch and features state-of-theart HD/VSA functionality, clear documentation, and implementation examples from wellknown publications. Comparing publicly available code with their corresponding Torchhd implementation shows that experiments can run up to 100x faster. Torchhd is available at: https://github.com/hyperdimensional-computing/torchhd. [ABSTRACT FROM AUTHOR]