Search

Brain-inspired approaches can efficiently analyze the activities of biological neural networks and solve computationally hard problems with energy efficiencies unattainable with von Neumann architectures, indicating a significant improvement in the understanding of neuronal communications and functionalities. Here, we present a brain-inspired multimodal signal processing system with organic memristor arrays that can potentially integrate the signal sensory, storage, and computation. To facilitate the multimodal signal processing system design, we used four components. First, we present a multimodal signal sensory module mainly responsible for multimodal (iconic, echoic, olfactory, muscular, and gustatory) signal collection, fusion, and storage. Second, a high-density cross-point memristive synapse array is constructed after fabrication of the albumin protein memristor to realize the dense connectivity between layers of computing, data storage, and communication. Third, considering the structure and function of the brain region, we demonstrate a general learning module for hierarchy learning, which can recognize and imagine multimodal information. Finally, the necessary peripheral circuit module (consisting of winnerless competition function circuit, analog-to-digital converter, digital-to-analog converter, pulse modulator, etc.) is designed. Notably, our system can capture massive amounts of data every second and perform in situ processing of multimodal signals. This study is expected to help achieve the deep integration of nano materials into neuromorphic computing systems and energy-efficient integrated circuits. [ABSTRACT FROM AUTHOR]

The biology-inspired intelligent computing system for the neuromorphic hardware implementation is useful in high-speed parallel information processing. However, the traditional Von Neumann computer architecture and the unsatisfactory signal transmission approach have jointly limited the overall performance of the specific hardware implementation. In this paper, a compact extreme learning machine (ELM) architecture synthesized with the spintronic memristor-based synaptic circuit, the biasing circuit, and the activation function circuit is presented. Notably, due to the threshold characteristic of the memristive device, the synaptic circuit has a bimodal behavior. Namely, it is capable to provide the constant and adjustable network weights between the adjacent layers in the ELM. Furthermore, two major limitations (process variations and sneak path issue) are taken into account for the detailed robustness analysis of the whole network. Finally, the entire scheme is verified with case studies in single image super-resolution (SR) reconstruction. [ABSTRACT FROM AUTHOR]

The nanoscale implementations of ternary logic circuits are particularly attractive because of high information density and operation speed that can be achieved by using emerging memristor technologies. Memristor is a nanoscale device with nonvolatility and adjustable multilevel states, which creates an intriguing opportunity for the implementation of ternary logic operations. This paper proposes a novel memristor-based design for stateful ternary logic, including AND, OR, NOT, NAND, NOR, and COPY operations. In the proposed memristor ternary logic (MTL) design, the resistance of memristor is the only logic state variable for representing the input and output. By sensing the value of the input memristors, the resistance of the output memristor changes accordingly. Furthermore, the MTL gates are not only capable of performing logic operations, but also storing logic values. To illustrate the potential of the methodology, a single-input-three-output ternary decoder is designed by using the proposed ternary logic circuits. Simulation results verify the effectiveness of the presented design. [ABSTRACT FROM AUTHOR]

The memristor is a nano-size electrical device that is widely investigated in memory, logic circuits, and neuromorphic systems. This paper puts forward a general window function for Hewlett-Packard (HP) TiO 2 memristor model, which covers almost all typical window functions and resolves the boundary lock problem, the inflexible parameter problem, and the distorted pinched hysteresis loop problem. Several adjustable parameters are introduced in the proposed window function in order to make it more fle x ible and scalable. A comparison with some classic window functions is given. Meanwhile, the range of these parameters is deduced with the aim of finding suitable parameters to simulate memristor devices. This TiO 2 memristor model based on the novel window function is further applied to the spiking neural network and tested with the MNIST database of handwritten digits. The result demonstrates that the memristor-based neural network has higher recognition accuracy and better classification effect, indicating the availability of the entire scheme. [ABSTRACT FROM AUTHOR]

Memristor is a novel passive electronic element with resistance-switching dynamics. Due to the threshold property and the variable conductivity of the memristive element, its composite circuits are promising for the implementation of logic operations. In this paper, a flexible logic circuit based on the threshold-type memristor and the mature complementary metal-oxide-semiconductor (CMOS) technology is designed for the realization of Boolean logic operations. Specifically, the proposed method is able to perform the NAND, AND, OR, and NOR gate operations through two phases, i.e. the writing operation and the reading operation. In such implementation, the total delay is very small especially for time-sequence inputs. Furthermore, for existing memristor-based logic implementation, a contrastive analysis with relevant computer simulations is carried out. The experimental results indicate that the proposed method is capable of realizing all basic Boolean logic operations, and some more complicated cascaded logic operations with more compact circuit structures, higher efficiency, and lower operating cost. [ABSTRACT FROM AUTHOR]

Pulse coupled neural network (PCNN) is a kind of visual cortex-inspired biological neural network, which has been proved a powerful candidate in the field of digital image processing due to its unique characteristics of global coupling and pulse synchronization. Notably, the inherent parameters estimation issue emerging in the entire system greatly affects the overall network performance. In this paper, a novel memristor crossbar array with its corresponding peripheral circuits is proposed, which is able to construct a general memristor-based PCNN (MPCNN) with variable linking coefficient. In order to verify the effectiveness and generality of the presented network, the single-channel MPCNN is further applied into the multi-focus image fusion problem with an improved multi-channel configuration. Correspondingly, a new type of MPCNN-based image fusion algorithm is put forward along with the design of an appropriate mapping function based on the image orientation information measure. Finally, a series of contrast experiments with comprehensive analysis demonstrate that the proposed fusion method has superior performances in terms of image quality and fusion effect compared to several existing algorithms. [ABSTRACT FROM AUTHOR]

A novel systematic design of associative memory networks is addressed in this paper, by incorporating both the biological small-world effect and the recently acclaimed memristor into the conventional Hopfield neural network. More specifically, the original fully connected Hopfield network is diluted by considering the small-world effect, based on a preferential connection removal criteria, i.e., weight salience priority. The generated sparse network exhibits comparable performance in associative memory but with much less connections. Furthermore, a hardware implementation scheme of the small-world Hopfield network is proposed using the experimental threshold adaptive memristor (TEAM) synaptic-based circuits. Finally, performance of the proposed network is validated by illustrative examples of digit recognition. [ABSTRACT FROM AUTHOR]

Traffic sign detection is a challenging task for the unmanned driving system, especially for the detection of multi-scale targets and the real-time problem of detection. In the traffic sign detection process, the scale of the targets changes greatly, which will have a certain impact on the detection accuracy. Feature pyramid is widely used to solve this problem, but due to the diversity of traffic sign sizes, it cannot accurately extract multi-size feature maps, thus destroying the feature consistency between traffic signs. Moreover, in practical application, it is difficult for common methods to improve the detection accuracy of multi-scale traffic signs while ensuring real-time detection. In this paper, we propose an improved feature pyramid model, named AF-FPN, which utilizes the adaptive attention module (AAM) and feature enhancement module (FEM) to reduce the information loss in the process of feature map generation and enhance the representation ability of the feature pyramid. We replaced the original feature pyramid network in YOLOv5 with AF-FPN, which improves the detection performance for multi-scale targets of the YOLOv5 network under the premise of ensuring real-time detection. Furthermore, a new automatic learning data augmentation method is proposed to enrich the dataset and improve the robustness of the model to make it more suitable for practical scenarios. Extensive experimental results on the Tsinghua-Tencent 100 K (TT100K) dataset demonstrate that compared with several state-of-the-art methods, our method is more universal and superior. [ABSTRACT FROM AUTHOR]

Reconfigurable nonvolatile logic within the crossbar array provides a promising approach for constructing a non-von Neumann architecture that integrates computation and storage. In this paper, a nonvolatile Boolean logic configuration based on one-transistor-one-memristor (1T1M) cells is presented. The proposed logic configuration is able to execute diverse Boolean logic operations simply by altering the applied voltage level. The logic design can be mapped to the 1T1M array, and the operation results can be stored in situ. Because of the high flexibility of the logic configuration, the 1T1M array can be configured to perform the encryption and decryption of binary images. The 1T1M array can be also used to realize composite logic functions. Furthermore, the impact of resistance variation on logic operations is taken into consideration, showing strong robustness. Simulation results demonstrate the feasibility of the proposed reconfigurable nonvolatile logic design. [ABSTRACT FROM AUTHOR]

The rapid development of smart factories, combined with the increasing complexity of production equipment, has resulted in a large number of multivariate time series that can be recorded using sensors during the manufacturing process. The anomalous patterns of industrial production may be hidden by these time series. Previous LSTM-based and machine-learning-based approaches have made fruitful progress in anomaly detection. However, these multivariate time series anomaly detection algorithms do not take into account the correlation and time dependence between the sequences. In this study, we proposed a new algorithm framework, namely, graph attention network and temporal convolutional network for multivariate time series anomaly detection (GTAD), to address this problem. Specifically, we first utilized temporal convolutional networks, including causal convolution and dilated convolution, to capture temporal dependencies, and then used graph neural networks to obtain correlations between sensors. Finally, we conducted sufficient experiments on three public benchmark datasets, and the results showed that the proposed method outperformed the baseline method, achieving detection results with F1 scores higher than 95% on all datasets. [ABSTRACT FROM AUTHOR]

Visual target tracking remains a hard research problem due to many unpredictable challenges (e.g., complete occlusion, out-of-view, and fast motion). This paper proposes a novel kernelized correlation filter architecture based on the fusion of the multiple feature response maps (FRMs), enhanced target re-detection, and improved online model updating. First, the multi-feature fusion strategy reveals that complementary feature information paves the way for effectively improving the target recognition ability. Meanwhile, a novel target re-detection module performs superior in relocating the tracked target and mitigating the model drift problem. Moreover, most existing trackers may indiscriminately update the model when encountering inaccurate predictions, eventually induce undesired model drift and tracking performance decay. Inspiring by the average peak-to-correlation energy, we apply an improved model updating scheme to ascertain the relative fluctuated level of FRMs. This novel model updating method can reject deteriorated samples and improves the discriminative power of the model. As evaluated on the four popular tracking benchmark datasets (including OTB-2013, OTB-2015, UAV123, and LaSOT), benefiting from proposed methods, our tracker obtains better effectiveness and efficiency than that of 28 state-of-the-art competitors (including 17 correlation filter-based competitors and 11 deep learning-based competitors). Meanwhile, our proposed tracker can meet the requirements of limited storage space and real-time performance. [ABSTRACT FROM AUTHOR]

Intelligent Transportation System (ITS) aims to strengthen the connection between vehicles, roads, and people. As the important road information in ITS, intelligent detection of traffic signs has become an important part in the intelligent vehicle. In this paper, a lightweight vehicle mounted multi-scale traffic sign detector is proposed. First, guided by the attention fusion algorithm, an improved feature pyramid network is proposed, named AFPN. Assign weights according to the importance of information and fuse multi-dimensional attention maps to improve feature extraction and information retention capabilities. Second, a multi-head detection structure is designed to improve the positioning and detection capability of the detector. According to the target scale, the corresponding detection head is constructed to improve the target detection accuracy. The experimental results show that compared with other state-of-the-art methods, the proposed method not only has excellent detection accuracy with 50.3% for small targets and 64.8% for large targets but also can better trade-off detection speed and detection accuracy. Furthermore, the proposed detector is deployed on the Jetson Xavier NX and integrated with the vehicle-mounted camera, inverter, and LCD to realize real-time traffic sign detection on the vehicle terminal, and the speed reaches 25.6 FPS. [ABSTRACT FROM AUTHOR]

Memristive technologies are attractive due to their nonvolatility, high density, low power, nanoscale geometry, nonlinearity, binary/multiple memory capacity, and negative differential resistance. For memristive devices, a model corresponding with practical behavioral characteristics is highly favorable for the realization of its neuromorphic system and applications. In this paper, we propose a novel memristor model based on the Ag/TiOx nanobelt/Ti configuration, which can reflect three different states (i.e. original stage, transition stage, and resistive switching state) of the physical memristor with a satisfactory fitting precision (greater than 99.88%). Meanwhile, this work gives (1) an insight onto the electrical characteristics of the memristor model under different humidity conditions; (2) the influence of the water molecular concentration on the memristor behavior, which is of importance for the memristor fabrication and subsequent applications. For verification purposes, the proposed three-state switchable memristor is applied into the memristor-based logic implementation. The experimental results demonstrate that the constructed circuit is able to realize basic Boolean logic operations with fast response speed and high efficiency. [ABSTRACT FROM AUTHOR]

Aiming at the current problem of insufficient extraction of small retinal blood vessels, we propose a retinal blood vessel segmentation algorithm that combines supervised learning and unsupervised learning algorithms. In this study, we use a multiscale matched filter with vessel enhancement capability and a U-Net model with a coding and decoding network structure. Three channels are used to extract vessel features separately, and finally, the segmentation results of the three channels are merged. The algorithm proposed in this paper has been verified and evaluated on the DRIVE, STARE, and CHASE_DB1 datasets. The experimental results show that the proposed algorithm can segment small blood vessels better than most other methods. We conclude that our algorithm has reached 0.8745, 0.8903, and 0.8916 on the three datasets in the sensitivity metric, respectively, which is nearly 0.1 higher than other existing methods. [ABSTRACT FROM AUTHOR]

Monocular depth estimation makes it possible for machines to perceive the real world. The prediction performance of the depth estimation network based on deep learning will be affected due to the depth of the deep network and the locality of convolution operations. The imitation of the biological visual system and its functional structure is becoming a research hotspot. In this paper, we study the interpretability relationship between the biological visual system and the monocular depth estimation network. By concretizing the attention mechanism in biological vision, we propose a monocular depth estimation network based on the self-attention mechanism, named SABV-Depth, which can improve prediction accuracy. Inspired by the biological visual interaction mechanism, we focus on the information transfer between each module of the network and improve the information retention ability, and enable the network to output a depth map with rich object information and detailed information. Further, a decoder module with an inner-connection is proposed to recover depth maps with sharp edge contours. Our method is experimentally validated on the KITTI dataset and NYU Depth V2 dataset. The results show that compared with other works, the proposed method improves prediction accuracy. Meanwhile, the depth map has more object information and detail information, and a better edge information processing effect. [ABSTRACT FROM AUTHOR]

With the widespread use of deep learning in single object tracking task, mainstream tracking algorithms treat tracking as a combined classification and regression problem. Classification aims at locating an arbitrary target, and regression aims at estimating the corresponding bounding box. In this paper, we focus on regression and propose a novel box estimation network, which consists of a transformer encoder target pyramid guide (TPG) and transformer decoder target pyramid spread (TPS). Specifically, the transformer encoder TPG is designed to generate fine-grained prior knowledge with explicit representation for template targets. In contrast to the raw transformer encoder, we capture the visual dependence through local-global self-attention and deem the multi-scale target regions as the “local” region. Using this fine-grained prior knowledge, we design the transformer decoder TPS to spread it to the subsequent search regions with high affinity to accurately estimate the bounding boxes. Considering that self-attention fails to model information interaction across channels between the template target and search regions, we develop a channel-wise cross-attention block within the TPS as compensation. Extensive experiments on the OTB100, UAV123, NFS, VOT2020, VOT2021, LaSOT, LaSOT_ext, TrackingNet and GOT-10k benchmarks show that the proposed box estimation network outperforms most existing box estimation methods. Furthermore, our trackers based on this estimation network exhibit a competitive performance against state-of-the-art trackers. [ABSTRACT FROM AUTHOR]

Multi-source domain adaptation, an effective solution for rotating machinery fault diagnosis, has achieved great success. However, previous multi-source domain adaptation based methods typically achieve domain consistency by aligning the global distribution of the source and target domains, regardless of private domain information and sufficient feature transferability, thereby limiting the diagnosis performance in cross-domain diagnostic scenarios. To address these issues, we propose a novel fine-grained feature decoupling based multi-source domain adaptation network (FD-MDAN) for fault diagnosis. Leveraging fine-grained feature decoupling, our FD-MDAN weakens specific features with private domain information and reinforces invariant features with diverse transferability at both of domain and category levels. We further develop a diversity regular term to minimize the feature redundancy, enhancing the transferability. In addition, to ensure that these decoupled features retain representative information of the original data, a reconstruction module is designed to make the reconstructed data and the original data consistent. Extensive experiments are conducted on three rotating machinery datasets. Benefiting from these components, our proposed FD-MDAN demonstrates state-of-the-art performance on various transfer diagnosis tasks. • A novel FD-MDAN is proposed for rotating machinery fault diagnosis under variable operating conditions. • Diverse multi-domain transferability is greatly enhanced for better generalization. • A fine-grained feature decoupling module is proposed for alleviating domain distribution shifts. • A diversity regular term is designed to minimize the feature redundancy. • Excellent performance in complex diagnosis scenarios is verified. [ABSTRACT FROM AUTHOR]

This paper presents a general modeling method for threshold-type multivalued memristors. Through this memristor modeling method, it is very simple to establish threshold-type memristor behavior models with different numbers of memristance elements, and these models are verified by numerical MATLAB simulations. A corresponding circuit-level SPICE model of the ternary memristor behavior model is developed and simulated in LTspice, shown to be consistent with the MATLAB results. Finally, the SPICE model is used to design the AND gate, OR gate, and three NOT gates of ternary state-based logic, and the effectiveness of the circuit is proved by LTSpice simulation. [ABSTRACT FROM AUTHOR]

• A progressive multi-source domain adaptation method is proposed for bearing fault diagnosis. • Source-source and source-target domain pairs are mapped to different feature spaces for domain pairwise alignment by LMMD metric. • Category confusion constraint is applied to reduce ambiguous samples. • Through a progressive optimisation process, domain-uninformative and category-informative features are guided. Based on massive samples collected from various working conditions, multi-source domain adaptation-based fault diagnosis methods have been a promising way to improve the generalization capability of diagnostic models. However, insufficient multi-domain alignment and category confusion issues lead to unsatisfactory diagnostic performance in practical cross-domain scenarios. This article proposes a progressive multi-source domain adaptation method, which learns domain-uninformative and category-informative features. First, all domains are divided into multiple source-source and source-target domain pairs on different feature spaces to explore rich domain-invariant features, where local maximum mean discrepancy is introduced for domain pairwise alignment. Second, category correlation is reduced to allow ambiguous samples to obtain clear category boundaries. Extensive experimental results show the outstanding performance of our method. [ABSTRACT FROM AUTHOR]

In recent years, convolution operations often consume a lot of time and energy in deep learning algorithms, and convolution is usually used to remove noise or extract the edges of an image. However, under data-intensive conditions, frequent operations of the above algorithms will cause a significant memory/communication burden to the computing system. This paper proposes a circuit based on spin memristor cross array to solve the problems mentioned above. First, a logic switch based on spin memristors is proposed, which realizes the control of the memristor cross array. Secondly, a new type of spin memristor cross array and peripheral circuits is proposed, which realizes the multiplication and addition operation in the convolution operation and significantly alleviates the computational memory bottleneck. At last, the color image filtering and edge extraction simulation are carried out. By calculating the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) of the image result, the processing effects of different operators are compared, and the correctness of the circuit is verified. [ABSTRACT FROM AUTHOR]

In order to improve the efficiency and reduce the labor cost of cafeterias, an intelligent master–slave collaborative robot system is developed for cafeteria service in this paper. The developed system can automatically complete the tasks of scooping dishes, taking bowls and pouring dishes into the bowl based on master–slave collaboration. Specifically, a dynamic geometry feature graph convolution network (DGG) is devised using the 3D point cloud of the dish, which can efficiently predict the scooping positions of the different dishes. Moreover, a master–slave motion planning control method is proposed to achieve fast and smooth trajectories for both arms, which can accomplish the cafeteria service tasks collaboratively. Furthermore, we establish a dataset containing point clouds and color images of various Chinese food. Experiments demonstrate that the DGG network can achieve superior performance over other state-of-the-art point cloud segmentation networks. Besides, the designed robot system can well meet the requirements of operation accuracy and speed, confirming its practicality in cafeteria services. • An intelligent master–slave collaborative robot system is developed for cafeteria service. • A novel point cloud segmentation network DGG is devised to predict the scooping positions of different dishes. • Using the master–slave motion planning, the proposed system can realize efficient and close coordination. [ABSTRACT FROM AUTHOR]

In the application of a long series battery group, an inter-pack imbalance is inevitable. No intra-pack cell equalizer can prevent pack-level over-discharge. A bimodal, multichannel battery pack equalizer based on a quasi-resonant, two-transistor forward converter is proposed to solve this problem and achieve a tradeoff between balancing efficiency and speed. This equalizer has two modes: pack-to-pack-group and pack-to-any-pack (P2PG&AP) mode and direct-pack-to-pack (DP2P) mode. In P2PG&AP mode, this equalizer can realize the full-switching-cycle (FSC) equalization through three balancing channels, and transfer energy from any pack to both the whole group and any pack inside the group. In addition, it can effectively clamp the transformer-induced voltage using a secondary side two-transistor magnetic reset structure (STMR) and reduce the total turns of transformer coil from 70 to 50 turns via a secondary side boost converter (SBC). In DP2P mode, this equalizer can realize zero voltage gap (ZVG) equalization. A prototype was tested at different switching frequencies and LC values to validate the theoretical analysis and optimize the bimodal hybrid operation. Experiment results including higher than 89.66% efficiency and minute-level balancing time under different pack voltage distributions show that the proposed topology demonstrates excellent balancing performance. [ABSTRACT FROM AUTHOR]

Effective traffic sign recognition algorithms can assist drivers or automatic driving systems in detecting and recognizing traffic signs in real-time. This paper proposes a multiscale recognition method for traffic signs based on the Gaussian Mixture Model (GMM) and Category Quality Focal Loss (CQFL) to enhance recognition speed and recognition accuracy. Specifically, GMM is utilized to cluster the prior anchors, which are in favor of reducing the clustering error. Meanwhile, considering the most common issue in supervised learning (i.e., the imbalance of data set categories), the category proportion factor is introduced into Quality Focal Loss, which is referred to as CQFL. Furthermore, a five-scale recognition network with a prior anchor allocation strategy is designed for small target objects i.e., traffic sign recognition. Combining five existing tricks, the best speed and accuracy tradeoff on our data set (40.1% mAP and 15 FPS on a single 1080Ti GPU), can be achieved. The experimental results demonstrate that the proposed method is superior to the existing mainstream algorithms, in terms of recognition accuracy and recognition speed. [ABSTRACT FROM AUTHOR]

Although correlation filter-based trackers (CFTs) have made great achievements on both robustness and accuracy, the performance of trackers can still be improved, because most of the existing trackers use either a sole filter template or fixed features fusion weight to represent a target. Herein, a real-time dual-template CFT for various challenge scenarios is proposed in this work. First, the color histograms, histogram of oriented gradient (HOG), and color naming (CN) features are extracted from the target image patch. Then, the dual-template is utilized based on the target response confidence. Meanwhile, in order to solve the various appearance variations in complicated challenge scenarios, the schemes of discriminative appearance model, multi-peaks target re-detection, and scale adaptive are integrated into the proposed tracker. Furthermore, the problem that the filter model may drift or even corrupt is solved by using high confidence template updating technique. In the experiment, 27 existing competitors, including 16 handcrafted features-based trackers (HFTs) and 11 deep features-based trackers (DFTs), are introduced for the comprehensive contrastive analysis on four benchmark databases. The experimental results demonstrate that the proposed tracker performs favorably against state-of-the-art HFTs and is comparable with the DFTs. [ABSTRACT FROM AUTHOR]