Search

Hyperspectral remote sensing technology is swiftly evolving, prioritizing affordability, enhanced portability, seamless integration, sophisticated intelligence, and immediate processing capabilities. The leading model for classifying hyperspectral images, which relies on convolutional neural networks (CNNs), has proven to be highly effective when run on advanced computing platforms. Nonetheless, the high degree of parameterization inherent in CNN models necessitates considerable computational and storage resources, posing challenges to their deployment in processors with limited capacity like drones and satellites. This paper focuses on advancing lightweight models for hyperspectral image classification and introduces EBCNN, a novel binary convolutional neural network. EBCNN is designed to effectively regulate backpropagation gradients and minimize gradient discrepancies to optimize BNN performance. EBCNN incorporates an adaptive gradient scaling module that utilizes a multi-scale pyramid squeeze attention (PSA) mechanism during the training phase, which can adjust training gradients flexibly and efficiently. Additionally, to address suboptimal training issues, EBCNN employs a dynamic curriculum learning strategy underpinned by a confidence-aware loss function, Superloss, enabling progressive binarization and enhancing its classification effectiveness. Extensive experimental evaluations conducted on five esteemed public datasets confirm the effectiveness of EBCNN. These analyses highlight a significant enhancement in the classification accuracy of hyperspectral images, achieved without incurring additional memory or computational overheads during the inference process. [ABSTRACT FROM AUTHOR]

The in-memory computing (IMC) architecture implemented by non-volatile memory units shows great possibilities to break the traditional von Neumann bottleneck. In this paper, a 3D IMC architecture is proposed whose unit is based on a multi-bit content-addressable memory (MCAM). The MCAM unit is comprised of two 65 nm flash memory and two transistors (2Flash2T), which is reconfigurable and multifunctional for both data write/search and XNOR logic operation. Moreover, the MCAM array can also support the population count (POPCOUNT) operation, which can be beneficial for the training and inference process in binary neural network (BNN) computing. Based on the well-known MNIST dataset, the proposed 3D MCAM architecture shows a 98.63% recognition accuracy and a 300% noise-tolerant performance without significant accuracy deterioration. Our findings can provide the potential for developing highly energy-efficient BNN computing for complex artificial intelligence (AI) tasks based on flash-based MCAM units. [ABSTRACT FROM AUTHOR]

Copyright of Systems Engineering & Electronics is the property of Journal of Systems Engineering & Electronics Editorial Department and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

This paper proposes the "ultimate compression" method as a solution to the expansive computation and high storage costs required by state-of-the-art neural network models in inference. Our approach uniquely combines tensor decomposition techniques with binary neural networks to create efficient deep neural network models optimized for edge inference. The process includes training floating-point models, applying tensor decomposition algorithms, binarizing the decomposed layers, and fine tuning the resulting models. We evaluated our approach in various state-of-the-art deep neural network architectures on multiple datasets, such as MNIST, CIFAR-10, CIFAR-100, and ImageNet. Our results demonstrate compression ratios of up to 169×, with only a small degradation in accuracy (1–2%) compared to binary models. We employed different optimizers for training and fine tuning, including Adam and AdamW, and used norm grad clipping to address the exploding gradient problem in decomposed binary models. A key contribution of this work is a novel layer sensitivity-based rank selection algorithm for tensor decomposition, which outperforms existing methods such as random selection and Variational Bayes Matrix Factorization (VBMF). We conducted comprehensive experiments using six different models and present a case study on crowd-counting applications, demonstrating the practical applicability of our method. The ultimate compression method outperforms binary neural networks and tensor decomposition when applied individually in terms of storage and computation costs. This positions it as one of the most effective options for deploying compact and efficient models in edge devices with limited computational resources and energy constraints. [ABSTRACT FROM AUTHOR]

Computational imaging, as a novel technology utilizing encoded image acquisition, relies on intelligent decoding methods for effective image restoration and sensing. Optical computing‐based decoders can efficiently process and extract features from pre‐sensor information, reducing the computational burden on digital computers. However, mainstream parallel optical neural network (ONN) architectures based on wavefront propagation typically possess complex network structures and high‐precision parameters, which pose challenges in terms of precise fabrication and system calibration, as well as sensitivity to signal‐to‐noise ratios. In this work, a binary‐weighted optical computing engine is proposed with spatial multiplexing and aggregation (B‐OSMA), a large‐scale passive ONN implementation that achieves high‐efficiency image sensing. Employing B‐OSMA as an optical decoder, demonstrated image categorizing from 2% compressive is experimented sampling with 92.0% and 83.8% accuracy on MNIST and fashion‐MNIST datasets, respectively, approaching the performance of full‐precision electronic computing while reducing storage requirements by 97%. Compared to conventional ONNs with analog weights, the B‐OSMA exhibits enhanced resilience against systematic errors and ambient noise. This work represents a significant advancement towards practical applications of optical computing in image sensing. [ABSTRACT FROM AUTHOR]

The evolution of super-resolution (SR) technology has seen significant advancements through the adoption of deep learning methods. However, the deployment of such models by resource-constrained devices necessitates models that not only perform efficiently, but also conserve computational resources. Binary neural networks (BNNs) offer a promising solution by minimizing the data precision to binary levels, thus reducing the computational complexity and memory requirements. However, for BNNs, an effective architecture is essential due to their inherent limitations in representing information. Designing such architectures traditionally requires extensive computational resources and time. With the advancement in neural architecture search (NAS), differentiable NAS has emerged as an attractive solution for efficiently crafting network structures. In this paper, we introduce a novel and efficient binary network search method tailored for image super-resolution tasks. We adapt the search space specifically for super resolution to ensure it is optimally suited for the requirements of such tasks. Furthermore, we incorporate Libra Parameter Binarization (Libra-PB) to maximize information retention during forward propagation. Our experimental results demonstrate that the network structures generated by our method require only a third of the parameters, compared to conventional methods, and yet deliver comparable performance. [ABSTRACT FROM AUTHOR]

Binarization is an extreme quantization technique that is attracting research in the Internet of Things (IoT) field, as it radically reduces the memory footprint of deep neural networks without a correspondingly significant accuracy drop. To support the effective deployment of Binarized Neural Networks (BNNs), we propose CBin-NN, a library of layer operators that allows the building of simple yet flexible convolutional neural networks (CNNs) with binary weights and activations. CBin-NN is platform-independent and is thus portable to virtually any software-programmable device. Experimental analysis on the CIFAR-10 dataset shows that our library, compared to a set of state-of-the-art inference engines, speeds up inference by 3.6 times and reduces the memory required to store model weights and activations by 7.5 times and 28 times, respectively, at the cost of slightly lower accuracy (2.5%). An ablation study stresses the importance of a Quantized Input Quantized Kernel Convolution layer to improve accuracy and reduce latency at the cost of a slight increase in model size. [ABSTRACT FROM AUTHOR]

In the realm of neuromorphic computing, integrating Binary Neural Networks (BNN) with non‐volatile memory based on emerging materials can be a promising avenue for introducing novel functionalities. This study underscores the viability of lead‐free, air‐stable Cs2SnI6 (CSI) based resistive random access memory (RRAM) devices as synaptic weights in neuromorphic architectures, specifically for BNNs applications. Herein, hydrothermally synthesized CSI perovskites are explored as a resistive layer in RRAM devices either on the rigid or flexible substrate, highlighting reproducible multibit switching with self‐compliance, low‐ resistance‐state (LRS) variations, a decent On/Off ratio(or retention) of ≈103(or 104 s), and endurance exceeding 300 cycles. Moreover, a comprehensive evaluation with the 32 × 32 × 3 RGB CIFAR‐10 dataset reveals that binary convolutional neural networks (BCNN) trained solely on binary weight values can achieve competitive rates of accuracy comparable to those of their analog weight counterparts. These findings highlight the dominance of the LRS for CSI RRAM with self‐compliance in a weighted configuration and minimal influence of the high resistance state despite substantial fluctuations for flexible CSI RRAM under varying bending radii. With its unique electrical switching capabilities, the CSI RRAM is highly anticipated to emerge as a promising candidate for embedded AI systems, especially in IoT devices and wearables. [ABSTRACT FROM AUTHOR]

This paper describes an in-memory computing architecture that combines full-precision computation for the first and last layers of a neural network while employing binary weights and input activations for the intermediate layers. This unique approach presents an efficient and effective solution for optimizing neural-network computations, reducing complexity, and enhancing energy efficiency. Notably, multiple architecture-level optimization methods are developed to ensure the binary operations thereby eliminating the need for intricate “digital logic” components external to the memory units. One of the key contributions of this study is in-memory batch normalization, which is implemented to provide good accuracy for CIFAR10 classification applications. Despite the inherent challenges posed by the process variations, the proposed design demonstrated an accuracy of 78%. Furthermore, the SRAM layer in the architecture showed an energy efficiency of 1086 TOPS/W and throughput of 23 TOPS, all packed efficiently within an area of 60 TOPS/mm2. This novel in-memory computing architecture offers a promising solution for next-generation efficient and high-performance deep learning applications.

This paper proposes the “ultimate compression” method as a solution to the expansive computation and high storage costs required by state-of-the-art neural network models in inference. Our approach uniquely combines tensor decomposition techniques with binary neural networks to create efficient deep neural network models optimized for edge inference. The process includes training floating-point models, applying tensor decomposition algorithms, binarizing the decomposed layers, and fine tuning the resulting models. We evaluated our approach in various state-of-the-art deep neural network architectures on multiple datasets, such as MNIST, CIFAR-10, CIFAR-100, and ImageNet. Our results demonstrate compression ratios of up to 169×, with only a small degradation in accuracy (1–2%) compared to binary models. We employed different optimizers for training and fine tuning, including Adam and AdamW, and used norm grad clipping to address the exploding gradient problem in decomposed binary models. A key contribution of this work is a novel layer sensitivity-based rank selection algorithm for tensor decomposition, which outperforms existing methods such as random selection and Variational Bayes Matrix Factorization (VBMF). We conducted comprehensive experiments using six different models and present a case study on crowd-counting applications, demonstrating the practical applicability of our method. The ultimate compression method outperforms binary neural networks and tensor decomposition when applied individually in terms of storage and computation costs. This positions it as one of the most effective options for deploying compact and efficient models in edge devices with limited computational resources and energy constraints.

In this article, an electrostatically-actuated NEMS XOR gate is proposed based on photonic crystals for hardware implementation of binary neural networks. The device includes a 2D photonic crystal which is set on a movable electrode to implement the XOR logic using the transmission of specific wavelengths to the output. This design represents the importance of the proposed structure in which the logic gate operation is not dependent on the contact of its conductive layers. Consequently, one of the major issues in MEMS-based logic gates, which is due to the contact of the operating electrodes and may cause stiction problem, reducing the reliability of the system, can be tackled by the present approach. Furthermore, according to the simulation results, the functional characteristics of the present NEMS XOR gate are obtained as follows: pull-in voltage of Vp = 8V, operating voltage of Vo = 10V and switching time of ts = 4 μs. The results also show that the proposed design provides a classification error rate of between 1% to 12%, while used in neural network implementation. This error can be negligible compared to the state-of-the-art designs in neural network implementation. These appropriate parameters of the present NEMS gate make it a promising choice for the implementation of neural networks with a high network accuracy even in the presence of significant process variations. [ABSTRACT FROM AUTHOR]

Neural architecture search (NAS) proves to be among the effective approaches for many tasks by generating an application-adaptive neural architecture, which is still challenged by high computational cost and memory consumption. At the same time, 1-bit convolutional neural networks (CNNs) with binary weights and activations show their potential for resource-limited embedded devices. One natural approach is to use 1-bit CNNs to reduce the computation and memory cost of NAS by taking advantage of the strengths of each in a unified framework, while searching the 1-bit CNNs is more challenging due to the more complicated processes involved. In this paper, we introduce Discrepant Child–Parent Neural Architecture Search (DCP–NAS) to efficiently search 1-bit CNNs, based on a new framework of searching the 1-bit model (Child) under the supervision of a real-valued model (Parent). Particularly, we first utilize a Parent model to calculate a tangent direction, based on which the tangent propagation method is introduced to search the optimized 1-bit Child. We further observe a coupling relationship between the weights and architecture parameters existing in such differentiable frameworks. To address the issue, we propose a decoupled optimization method to search an optimized architecture. Extensive experiments demonstrate that our DCP–NAS achieves much better results than prior arts on both CIFAR-10 and ImageNet datasets. In particular, the backbones achieved by our DCP–NAS achieve strong generalization performance on person re-identification and object detection. [ABSTRACT FROM AUTHOR]

Plant disease control has long been a critical issue in agricultural production and relies heavily on the identification of plant diseases, but traditional disease identification requires extensive experience. Most of the existing deep learning-based plant disease classification methods run on high-performance devices to meet the requirements for classification accuracy. However, agricultural applications have strict cost control and cannot be widely promoted. This paper presents a novel method for plant disease classification using a binary neural network with dual attention (DABNN), which can save computational resources and accelerate by using binary neural networks, and introduces a dual-attention mechanism to improve the accuracy of classification. To evaluate the effectiveness of our proposed approach, we conduct experiments on the PlantVillage dataset, which includes a range of diseases. The F 1 s c o r e and A c c u r a c y of our method reach 99.39% and 99.4%, respectively. Meanwhile, compared to AlexNet and VGG16, the C o m p u t a t i o n a l c o m p l e x i t y of our method is reduced by 72.3% and 98.7%, respectively. The P a r a m s s i z e of our algorithm is 5.4% of AlexNet and 2.3% of VGG16. The experimental results show that DABNN can identify various diseases effectively and accurately. [ABSTRACT FROM AUTHOR]

The ever-growing computational demands of increasingly complex machine learning models frequently necessitate the use of powerful cloud-based infrastructure for their training. Binary neural networks are known to be promising candidates for on-device inference due to their extreme compute and memory savings over higher-precision alternatives. However, their existing training methods require the concurrent storage of high-precision activations for all layers, generally making learning on memory-constrained devices infeasible. In this article, we demonstrate that the backward propagation operations needed for binary neural network training are strongly robust to quantization, thereby making on-the-edge learning with modern models a practical proposition. We introduce a low-cost binary neural network training strategy exhibiting sizable memory footprint reductions while inducing little to no accuracy loss vs Courbariaux & Bengio's standard approach. These decreases are primarily enabled through the retention of activations exclusively in binary format. Against the latter algorithm, our drop-in replacement sees memory requirement reductions of 3-5x, while reaching similar test accuracy (±2 pp) in comparable time, across a range of small-scale models trained to classify popular datasets. We also demonstrate from-scratch ImageNet training of binarized ResNet-18, achieving a 3.78x memory reduction. Our work is open-source, and includes the Raspberry Pi-targeted prototype we used to verify our modeled memory decreases and capture the associated energy drops. Such savings will allow for unnecessary cloud offloading to be avoided, reducing latency, increasing energy efficiency, and safeguarding end-user privacy. [ABSTRACT FROM AUTHOR]

Deep learning (DL) has recently changed the development of intelligent systems and is widely adopted in many real-life applications. Despite their various benefits and potentials, there is a high demand for DL processing in different computationally limited and energy-constrained devices. It is natural to study game-changing technologies such as Binary Neural Networks (BNN) to increase DL capabilities. Recently remarkable progress has been made in BNN since they can be implemented and embedded on tiny restricted devices and save a significant amount of storage, computation cost, and energy consumption. However, nearly all BNN acts trade with extra memory, computation cost, and higher performance. This article provides a complete overview of recent developments in BNN. This article focuses exclusively on 1-bit activations and weights 1-bit convolution networks, contrary to previous surveys in which low-bit works are mixed in. It conducted a complete investigation of BNN's development—from their predecessors to the latest BNN algorithms/techniques, presenting a broad design pipeline and discussing each module's variants. Along the way, it examines BNN (a) purpose: their early successes and challenges; (b) BNN optimization: selected representative works that contain essential optimization techniques; (c) deployment: open-source frameworks for BNN modeling and development; (d) terminal: efficient computing architectures and devices for BNN and (e) applications: diverse applications with BNN. Moreover, this paper discusses potential directions and future research opportunities in each section. [ABSTRACT FROM AUTHOR]

This paper proposes a Reconfigurable In-Memory Advance Computing architecture using a novel 10 SRAM cell. In addition to basic logic operations, the proposed R-InMAC can also implement complex Boolean computing operations such as binary addition/subtraction, binary-to-gray, gray-to-binary conversion, 2's complement, less/greater than, and increment/decrement. Furthermore, content addressable memory (CAM) operation to search a binary string in a memory array is also proposed efficiently. It can search true and complementary data strings in a single cycle. The proposed R-InMAC architecture's reconfigurability allows it to be configured according to the needed operation and bit precision, making it ideal and energy-efficient. In addition, compared to the standard SRAM cells, the proposed 10T cell is suited for implementing the XNOR-based binary convolution operation required in Binary Neural Networks (BNNs) with improved latency of 58.89%. The optimized full adder of the proposed R-InMAC shows decrement in the area by 40%, static power by 28%, dynamic power by 55.2%, and latency by 25.3% as compared to conventional designs, making this work a promising candidate for modern edge AI compute in-memory systems. [ABSTRACT FROM AUTHOR]

This paper presents an extensive literature review on Binary Neural Network (BNN). BNN utilizes binary weights and activation function parameters to substitute the full-precision values. In digital implementations, BNN replaces the complex calculations of Convolutional Neural Networks (CNNs) with simple bitwise operations. BNN optimizes large computation and memory storage requirements, which leads to less area and power consumption compared to full-precision models. Although there are many advantages of BNN, the binarization process has a significant impact on the performance and accuracy of the generated models. To reflect the state-of-the-art in BNN and explore how to develop and improve BNN-based models, we conduct a systematic literature review on BNN with data extracted from 239 research studies. Our review discusses various BNN architectures and the optimization approaches developed to improve their performance. There are three main research directions in BNN: accuracy optimization, compression optimization, and acceleration optimization. The accuracy optimization approaches include quantization error reduction, special regularization, gradient error minimization, and network structure. The compression optimization approaches combine fractional BNN and pruning. The acceleration optimization approaches comprise computing in-memory, FPGA-based implementations, and ASIC-based implementations. At the end of our review, we present a comprehensive analysis of BNN applications and their evaluation metrics. Also, we shed some light on the most common BNN challenges and the future research trends of BNN.

This study aims to leverage Binary Neural Networks (BNN) to learn binary hash codes for efficient person re-identification (ReID). BNNs, which use binary weights and activations, show promise in speeding up the inference time in deep models. However, BNNs typically suffer from performance degradation mainly due to the discontinuity of the binarization operation. Proxy functions have been proposed to calculate the gradients in the backward propagation, but they lead to the gradient mismatch problem. In this study, we propose to address the gradient mismatch problem by designing a multi-branch ensemble model consisting of many weak hash code learners. Specifically, our design aggregates the gradients from multiple branches, which allows a better approximation of the gradients and regularizes the network. Our model adds little computational cost to the baseline BNN since a vast amount of network parameters are shared between the weak learners. Combining the efficiency of the BNNs and hash code learning, we obtain an effective ensemble model which is efficient both in feature extraction and ranking phases. Our experiments demonstrate that the proposed model outperforms a single BNN by more than %20 using nearly the same amount of floating point operations. Moreover, the proposed model outperforms a conventional ensemble of BNN by more than %7 while being nearly 10x and 2x more efficient in terms of CPU consumption and memory footprint, respectively. We explore the performance of BNNs for efficient person ReID as one of the first systems available in the literature. Moreover, we adopt the proposed ensemble model for further validation of the image classification task and observe that our method effectively regularizes BNNs, providing robustness to hyperparameter selection and producing more consistent results under different settings.

Remarkable achievements have been obtained with binary neural networks (BNN) in real-time and energy-efficient single-image super-resolution (SISR) methods. However, existing approaches often adopt the Sign function to quantize image features while ignoring the influence of image spatial frequency. We argue that we can minimize the quantization error by considering different spatial frequency components. To achieve this, we propose a frequency-aware binarized network (FABNet) for single image super-resolution. First, we leverage the wavelet transformation to decompose the features into low-frequency and high-frequency components and then employ a “divide-and-conquer” strategy to separately process them with well-designed binary network structures. Additionally, we introduce a dynamic binarization process that incorporates learned-threshold binarization during forward propagation and dynamic approximation during backward propagation, effectively addressing the diverse spatial frequency information. Compared to existing methods, our approach is effective in reducing quantization error and recovering image textures. Extensive experiments conducted on four benchmark datasets demonstrate that the proposed methods could surpass state-of-the-art approaches in terms of PSNR and visual quality with significantly reduced computational costs. Our codes are available at https://github.com/xrjiang527/FABNet-PyTorch. [ABSTRACT FROM AUTHOR]

Binary neural network (BNN) provides a promising solution to deploy parameter-intensive deep single image super-resolution (SISR) models onto real devices with limited storage and computational resources. To achieve comparable performance with the full-precision counterpart, most existing BNNs for SISR mainly focus on compensating for the information loss incurred by binarizing weights and activations in the network through better approximations to the binarized convolution. In this study, we revisit the difference between BNNs and their full-precision counterparts and argue that the key to good generalization performance of BNNs lies on preserving a complete full-precision information flow along with an accurate gradient flow passing through each binarized convolution layer. Inspired by this, we propose to introduce a full-precision skip connection, or a variant thereof, over each binarized convolution layer across the entire network, which can increase the forward expressive capability and the accuracy of back-propagated gradient, thus enhancing the generalization performance. More importantly, such a scheme can be applied to any existing BNN backbones for SISR without introducing any additional computation cost. To validate the efficacy of the proposed approach, we evaluate it using four different backbones for SISR on four benchmark datasets and report obviously superior performance over existing BNNs and even some 4-bit competitors. [ABSTRACT FROM AUTHOR]

The outputs of deep hash network (DHN) are binary codes, so DHN has high retrieval efficiency in matching phase and can be used for high-speed palmprint recognition, which is a promising biometric modality. In this paper, the templates and network parameters are both quantized for fast and light-weight palmprint recognition. The parameters of DHN are binarized to compress the network weight and accelerate the speed. To avoid accuracy degradation caused by quantization, mutual information is leveraged to optimize the ambiguity in Hamming space to obtain a tri-valued hash code as a palmprint template. Kleene Logic's tri-valued Hamming distance measures the dissimilarity between palmprint templates. The ablation experiments are tested on the binarization of the network parameter, and the normalization and trivialization of the deep hash output value. The sufficient experiments conducted on several contact and contactless palmprint datasets confirm the multiple advantages of our method. [ABSTRACT FROM AUTHOR]

As the scales of neural networks increase, techniques that enable them to run with low computational cost and energy efficiency are required. From such demands, various efficient neural network paradigms, such as spiking neural networks (SNNs) or binary neural networks (BNNs), have been proposed. However, they have sticky drawbacks, such as degraded inference accuracy and latency. To solve these problems, we propose a single-step spiking neural network (S 3 NN), an energy-efficient neural network with low computational cost and high precision. The proposed S 3 NN processes the information between hidden layers by spikes as SNNs. Nevertheless, it has no temporal dimension so that there is no latency within training and inference phases as BNNs. Thus, the proposed S 3 NN has a lower computational cost than SNNs that require time-series processing. However, S 3 NN cannot adopt naïve backpropagation algorithms due to the non-differentiability nature of spikes. We deduce a suitable neuron model by reducing the surrogate gradient for multi-time step SNNs to a single-time step. We experimentally demonstrated that the obtained surrogate gradient allows S 3 NN to be trained appropriately. We also showed that the proposed S 3 NN could achieve comparable accuracy to full-precision networks while being highly energy-efficient. [ABSTRACT FROM AUTHOR]

Model binarization is an effective method of compressing neural networks and accelerating their inference process, which enables state-of-the-art models to run on resource-limited devices. Recently, advanced binarization methods have been greatly improved by minimizing the quantization error directly in the forward process. However, a significant performance gap still exists between the 1-bit model and the 32-bit one. The empirical study shows that binarization causes a great loss of information in the forward and backward propagation which harms the performance of binary neural networks (BNNs). We present a novel distribution-sensitive information retention network (DIR-Net) that retains the information in the forward and backward propagation by improving internal propagation and introducing external representations. The DIR-Net mainly relies on three technical contributions: (1) Information Maximized Binarization (IMB): minimizing the information loss and the binarization error of weights/activations simultaneously by weight balance and standardization; (2) Distribution-sensitive Two-stage Estimator (DTE): retaining the information of gradients by distribution-sensitive soft approximation by jointly considering the updating capability and accurate gradient; (3) Representation-align Binarization-aware Distillation (RBD): retaining the representation information by distilling the representations between full-precision and binarized networks. The DIR-Net investigates both forward and backward processes of BNNs from the unified information perspective, thereby providing new insight into the mechanism of network binarization. The three techniques in our DIR-Net are versatile and effective and can be applied in various structures to improve BNNs. Comprehensive experiments on the image classification and objective detection tasks show that our DIR-Net consistently outperforms the state-of-the-art binarization approaches under mainstream and compact architectures, such as ResNet, VGG, EfficientNet, DARTS, and MobileNet. Additionally, we conduct our DIR-Net on real-world resource-limited devices which achieves 11.1 × storage saving and 5.4 × speedup. [ABSTRACT FROM AUTHOR]

Molecular electronics exhibiting resistive-switching memory features hold great promise for the next generation of digital technology. In this work, electrosynthesis of ruthenium polypyridyl nanoscale oligomeric films is demonstrated on an indium tin oxide (ITO) electrode followed by an ITO top contact deposition yielding large-scale (junction area = 0.7 × 0.7 cm 2 ) two terminal molecular junctions. The molecular junctions exhibit non-volatile resistive switching at a relatively lower operational voltage, ±1 V, high ON/OFF electrical current ratio (≈10 3 ), low-energy consumption (SET/RESET = 27.94/14400 nJ), good cyclic stability (>300 cycles), and switching speed (SET/RESET = 25 ms/20 ms). A computational study suggests that accessible frontier molecular orbitals of metal-complex to the Fermi level of ITO electrodes facilitate charge transport at a relatively lower bias followed by a filamentformation. An extensive analysis is performed of the performance of binary neural networks exploiting the current-voltage features of the devices as binary synaptic weights and exploring their potential for neuromorphic logic-in-memory implementation of IMPLICATION (IMPLY) operation which can realize universal gates. The comprehensive analysis indicates that the proposed redox-active complex-based memory device may be a promising candidate for high-density data storage, energy-efficient implementation of neuromorphic networks with software-level accuracy, and logic-in-memory implementations., (© 2025 Wiley‐VCH GmbH.)

As the memory footprint requirement and computational scale concerned, the light-weighted Binary Neural Networks (BNNs) have great advantages in limited-resources platforms, such as AIoT (Artificial Intelligence in Internet of Things) edge terminals, wearable and portable devices, etc. However, the binarization process naturally brings considerable information losses and further deteriorates the accuracy. In this article, three aspects are introduced to better the binarized ReActNet accuracy performance with a more low-complex computation. Firstly, an improved Binarized Ghost Module (BGM) for the ReActNet is proposed to increase the feature maps information. At the same time, the computational scale of this structure is still kept at a very low level. Secondly, we propose a new Label-aware Loss Function (LLF) in the penultimate layer as a supervisor which takes the label information into consideration. This auxiliary loss function makes each category’s feature vectors more separate, and improve the final fully-connected layer’s classification accuracy accordingly. Thirdly, the Normalization-based Attention Module (NAM) method is adopted to regulate the activation flow. The module helps to avoid the gradient saturation problem. With these three approaches, our improved binarized network outperforms the other state-of-the-art methods. It can achieve 71.4% Top-1 accuracy on the ImageNet and 86.45% accuracy on the CIFAR-10 respectively. Meanwhile, its computational scale OPs is the least $0.86\times {10}^{8}$ compared with the other mainstream BNN models. The experimental results prove the effectiveness of our proposals, and the study is very helpful and promising for the future low-power hardware implementations.

A method for determining the weights of a binary neural network based on its decomposition into elementary modules is presented. The approach allows tuning the weight coefficients of all the network connections at the stage of its designing, which eliminates the implementation of time-consuming iterative algorithms for training the network during its operation. An algorithm and an example of calculating the weights are given. [ABSTRACT FROM AUTHOR]

In this paper, we find that the conventional convolution operation becomes the bottleneck for extremely efficient binary neural networks (BNNs). To address this issue, we open up a new direction by introducing a reshaped point-wise convolution (RPC) to replace the conventional one to build BNNs. Specifically, we conduct a point-wise convolution after rearranging the spatial information into depth, with which at least $2.25\times $ computation reduction can be achieved. Such an efficient RPC allows us to explore more powerful representational capacity of BNNs under a given computation complexity budget. Moreover, we propose to use a balanced activation (BA) to adjust the distribution of the scaled activations after binarization, which enables significant performance improvement of BNNs. After integrating RPC and BA, the proposed network, dubbed as RB-Net, strikes a good trade-off between accuracy and efficiency, achieving superior performance with lower computational cost against the state-of-the-art BNN methods. Specifically, our RB-Net achieves 66.8% Top-1 accuracy with ResNet-18 backbone on ImageNet, exceeding the state-of-the-art Real-to-Binary Net (65.4%) by 1.4% while achieving more than $3\times $ reduction (52M vs. 165M) in computational complexity. [ABSTRACT FROM AUTHOR]

Binary Neural Networks (BNNs) have gained extensive attention for their superior inferencing efficiency and compression ratio compared to traditional full-precision networks. However, due to the unique characteristics of BNNs, designing a powerful binary architecture is challenging and often requires significant manpower. A promising solution is to utilize Neural Architecture Search (NAS) to assist in designing BNNs, but current NAS methods for BNNs are relatively straightforward and leave a performance gap between the searched models and manually designed ones. To address this gap, we propose a novel neural architecture search scheme for binary neural networks, named NAS-BNN. We first carefully design a search space based on the unique characteristics of BNNs. Then, we present three training strategies, which significantly enhance the training of supernet and boost the performance of all subnets. Our discovered binary model family outperforms previous BNNs for a wide range of operations (OPs) from 20M to 200M. For instance, we achieve 68.20% top-1 accuracy on ImageNet with only 57M OPs. In addition, we validate the transferability of these searched BNNs on the object detection task, and our binary detectors with the searched BNNs achieve a novel state-of-the-art result, e.g. , 31.6% mAP with 370M OPs, on MS COCO dataset. The source code and models will be released at https://github.com/VDIGPKU/NAS-BNN. • A novel search space for binary neural networks is proposed. • Non-Decreasing (ND) constraint is proposed to remove the ineffective subnets, building a more powerful search space. • Three training techniques are proposed to enhance the training process of binary supernet, so as to boost the performance of each subnet. • Extensive experiments demonstrate that our method achieves better trade-offs than prior binary neural networks. [ABSTRACT FROM AUTHOR]

Significant volumes of wastewater are routinely generated during agro-industry processing, amounting to millions of tonnes annually. In line with the circular economy concept, there could be a possibility of simultaneously treating the wastewater and recovering bio-energy resources such as bio-hydrogen. This study aimed to model the effect of different process parameters that could influence wastewater treatment and bio-energy recovery from agro-industrial wastewaters. Three agro-industrial wastewaters from dairy, chicken processing, and palm oil mills were investigated. Eight data-driven machine learning algorithms namely linear support vector machine (LSVM), quadratic support vector machine (QSVM), cubic support vector machine (CSVM), fine Gaussian support vector machine (FGSVM), binary neural network (BNN), rotation quadratic Gaussian process regression (RQGPR), exponential quadratic Gaussian process regression (EQGPR) and exponential Gaussian process regression (EGPR) were employed for the modeling process. The datasets obtained from the three agro-industrial processes were employed to train and test the models. The LSVM, QSVM, and CSVM did not show an impressive performance as indicated by the coefficient of determination (R2) < 0.7 for the prediction of hydrogen produced from wastewaters using the three agro-industrial processes. The LSVM, QSVM, and CSVM models were also characterized by high prediction errors. Superior performance was displayed by FGSVM, BNN, RQGPR, EQGPR, and EQGPR models as indicated by the high R2 > 0.9, an indication of better predictability with minimized prediction errors as indicated by the low root mean square error (RMSE), mean square error (MSE), and mean absolute error (MAE).

Skeleton-based action recognition is an essential yet challenging visual task, whose accuracy has been remarkably improved due to the successful application of graph convolutional networks (GCNs). However, high computation cost and memory usage hinder their deployment on resource-constrained environment. To deal with the issue, in this paper, we introduce two novel progressive binary graph convolutional network for skeleton-based action recognition PB-GCN and PB-GCN * , which can obtain significant speed-up and memory saving. In PB-GCN, the filters are binarized, and in PB-GCN * , both filters and activations are binary. Specifically, we propose a progressive optimization, i.e., employing ternary models as the initialization of binary GCNs (BGCN) to improve the representational capability of binary models. Moreover, the center loss is exploited to improve the training procedure for better performance. Experimental results on two public benchmarks (i.e., Skeleton-Kinetics and NTU RGB + D) demonstrate that the accuracy of the proposed PB-GCN and PB-GCN * are comparable to their full-precision counterparts and outperforms the state-of-the-art methods, such as BWN, XNOR-Net, and Bi-Real Net. [ABSTRACT FROM AUTHOR]

A synaptic cell composed of two tunnel field-effect transistors (TFETs) which is capable of XNOR operation for binary neural networks has been experimentally demonstrated. Our proposed synaptic TFETs feature lower current during inference and higher programming efficiency during weight transfer than conventional synaptic transistors. Moreover, the fabricated synaptic TFET arrays satisfy the neurobiological energy requirement $(\!\sim 10$ fJ per synaptic event) and low bit-error rate of $6.7\times10$ −7%. [ABSTRACT FROM AUTHOR]

Enabling deep learning technology on the mobile web can improve the user’s experience for achieving web artificial intelligence in various fields. However, heavy DNN models and limited computing resources of the mobile web are now unable to support executing computationally intensive DNNs when deploying in a cloud computing platform. With the help of promising edge computing, we propose a lightweight collaborative deep neural network for the mobile web, named LcDNN, which contributes to three aspects: (1) We design a composite collaborative DNN that reduces the model size, accelerates inference, and reduces mobile energy cost by executing a lightweight binary neural network (BNN) branch on the mobile web. (2) We provide a jointly training method for LcDNN and implement an energy-efficient inference library for executing the BNN branch on the mobile web. (3) To further promote the resource utilization of the edge cloud, we develop a DRL-based online scheduling scheme to obtain an optimal allocation for LcDNN. The experimental results show that LcDNN outperforms existing approaches for reducing the model size by about 16x to 29x. It also reduces the end-to-end latency and mobile energy cost with acceptable accuracy and improves the throughput and resource utilization of the edge cloud. [ABSTRACT FROM AUTHOR]

End-to-end learning of the communication system regards the transmitter, channel, and receiver as a neural network-based autoencoder. This approach enables joint optimization of both the transmitter and receiver and can learn to communicate more efficiently than model-based ones. Despite the achieved success, high complexity is the major disadvantage that hinders its further development, while low-precision compression such as one-bit quantization is an effective solution. This study proposed an autoencoder communication system composed of binary neural networks (BNNs), which is based on bit operations and has a great potential to be applied to hardware platforms with very limited computing resources such as FPGAs. Several modifications are explored to further improve the performance. Experiments showed that the proposed BNN-based system can achieve a performance similar to that of the existing neural network-based autoencoder systems while largely reducing the storage and computation complexities.

Magnetic RAM (MRAM)-based crossbar array has a great potential as a platform for in-memory binary neural network (BNN) computing. However, the number of word-lines that can be activated simultaneously is limited because of the low $I_{H}/I_{L}$ ratio of MRAM, which makes BNNs more vulnerable to the device variation. To address this issue, we propose an algorithm/hardware co-design methodology. First, we choose a promising memristor crossbar array (MCA) structure based on the sensitivity analysis to process variations. Since the selected MCA structure becomes more tolerant to the device variation when the number of 1 in input activation values decreases, we apply an input distribution regularization scheme to reduce the number of 1 in input of BNNs during training. We further improve the robustness against device variation by adopting the retraining scheme based on knowledge distillation. Experimental results show that the proposed method makes BNNs more tolerant to MRAM variation and increases the number of parallel word-line activation significantly; thereby achieving improved throughput and energy efficiency.