Search

In recent years, plenty of metrics have been proposed to identify networks that are free of gradient explosion and vanishing. However, due to the diversity of network components and complex serial-parallel hybrid connections in modern DNNs, the evaluation of existing metrics usually requires strong assumptions, complex statistical analysis, or has limited application fields, which constraints their spread in the community. In this paper, inspired by the Gradient Norm Equality and dynamical isometry, we first propose a novel metric called Block Dynamical Isometry, which measures the change of gradient norm in individual block. Because our Block Dynamical Isometry is norm-based, its evaluation needs weaker assumptions compared with the original dynamical isometry. To mitigate the challenging derivation, we propose a highly modularized statistical framework based on free probability. Our framework includes several key theorems to handle complex serial-parallel hybrid connections and a library to cover the diversity of network components. Besides, several sufficient prerequisites are provided. Powered by our metric and framework, we analyze extensive initialization, normalization, and network structures. We find that Gradient Norm Equality is a universal philosophy behind them. Then, we improve some existing methods based on our analysis, including an activation function selection strategy for initialization techniques, a new configuration for weight normalization, and a depth-aware way to derive coefficients in SeLU. Moreover, we propose a novel normalization technique named second moment normalization, which is theoretically 30% faster than batch normalization without accuracy loss. Last but not least, our conclusions and methods are evidenced by extensive experiments on multiple models over CIFAR10 and ImageNet., Comment: Under review as a regular paper in TPAMI

Forest farms in Northeast China play an important role in achieving carbon peaking and carbon neutrality in the region. Balancing the various values of forests is important to achieve sustainable development in the region. In this paper, a CO2FIX model for calculating forest carbon sequestration is first established. Then an evaluation result-oriented decision model (ERDM) is established. Evaluate the carbon sequestration value, economic value and ecological value of forests, and propose oriented policies for the values with lower scores. Then a multi-objective planning process is established with the goal of maximizing the sum of various forest values, and the oriented policy directly affects the constraints. The results of multi-objective planning provide quantitative indicators for forest management plans. After testing, the above decision-making scheme has achieved the benign development of forest farms in Northeast China in the process of simulated management.

Deep neural networks (DNNs) thrive in recent years, wherein batch normalization (BN) plays an indispensable role. However, it has been observed that BN is costly due to the huge reduction and elementwise operations that are hard to be executed in parallel, which heavily reduces the training speed. To address this issue, in this article, we propose a methodology to alleviate the BN’s cost by using only a few sampled or generated data for mean and variance estimation at each iteration. The key challenge to reach this goal is how to achieve a satisfactory balance between normalization effectiveness and execution efficiency. We identify that the effectiveness expects less data correlation in sampling while the efficiency expects more regular execution patterns. To this end, we design two categories of approach: sampling or creating a few uncorrelated data for statistics’ estimation with certain strategy constraints. The former includes “batch sampling (BS)” that randomly selects a few samples from each batch and “feature sampling (FS)” that randomly selects a small patch from each feature map of all samples, and the latter is “virtual data set normalization (VDN)” that generates a few synthetic random samples to directly create uncorrelated data for statistics’ estimation. Accordingly, multiway strategies are designed to reduce the data correlation for accurate estimation and optimize the execution pattern for running acceleration in the meantime. The proposed methods are comprehensively evaluated on various DNN models, where the loss of model accuracy and the convergence rate are negligible. Without the support of any specialized libraries, $1.98\times $ BN layer acceleration and 23.2% overall training speedup can be practically achieved on modern GPUs. Furthermore, our methods demonstrate powerful performance when solving the well-known “micro-BN” problem in the case of a tiny batch size. This article provides a promising solution for the efficient training of high-performance DNNs.

Light detection and ranging (LiDAR) is a type of essential tool for urban planning and geoinformation extraction. Airborne streak tube imaging LiDAR (ASTIL) is a new system with great advantages in the rapid collection of remote sensing data. To the best of our knowledge, a new method to extract a building roof from the echo images of ASTIL is proposed. We improve YOLOv5s with a one-shot aggregation (OSA) module to improve efficiency. The experimental results show that the mean average precision of the OSA-YOLOv5s algorithm can reach 95.2%, and the frames per second can reach 11.74 using a CPU and 39.39 using a GPU. The method proposed can extract building objects efficiently from the echo images of ASTIL and acquire the building roof point cloud.

This paper describes a high-pulse-energy frequency-doubled ultraviolet dye laser operating at a repetition rate of 500 Hz. The pump source is a laser-diode side-pumped Q-switched Nd:YAG laser with ...

Transformers are becoming the mainstream solutions for various tasks like NLP and Computer vision. Despite their success, the high complexity of the attention mechanism hinders them from being applied to latency-sensitive tasks. Tremendous efforts have been made to alleviate this problem, and many of them successfully reduce the asymptotic complexity to linear. Nevertheless, most of them fail to achieve practical speedup over the original full attention under moderate sequence lengths and are unfriendly to finetuning. In this paper, we present DFSS, an attention mechanism that dynamically prunes the full attention weight matrix to N:M fine-grained structured sparse pattern. We provide both theoretical and empirical evidence that demonstrates DFSS is a good approximation of the full attention mechanism. We propose a dedicated CUDA kernel design that completely eliminates the dynamic pruning overhead and achieves speedups under arbitrary sequence length. We evaluate the 1:2 and 2:4 sparsity under different configurations and achieve 1.27~ 1.89x speedups over the full-attention mechanism. It only takes a couple of finetuning epochs from the pretrained model to achieve on par accuracy with full attention mechanism on tasks from various domains under different sequence lengths from 384 to 4096.

The success of DNN comes at the expense of excessive memory/computation cost, which can be addressed by exploiting reduced precision and sparsity jointly. Existing sparse GPU kernels, however, fail to achieve practical speedup over cuBLASHgemm under half-precision. Those for fine-grained sparsity suffer from low data reuse, and others for coarse-grained sparsity are limited by the wrestling between kernel performance and model quality under different grain sizes. We propose column-vector-sparse-encoding that has a smaller grain size under the same reuse rate compared with block sparsity. Column-vector-sparse-encoding can be applied to both SpMM & SDDMM, two major sparse DNN operations. We also introduce the Tensor-Core-based 1D Octet Tiling that has efficient memory access and computation patterns under small grain size. Based on these, we design SpMM and SDDMM kernels and achieve 1.71-7.19x speedup over cuSPARSE. Practical speedup is achieved over cuBLASHgemm under >70% and >90% sparsity with 4x1 grain size and half-precision.

In this paper, we present a dye laser operated under burst mode with minimum pulse interval 0.1 ms, corresponding to repetition rate 10 kHz. Pulse energy of 7.77 mJ with efficiency of 26.7% is achieved at 560 nm by using rhodamine 6G solution in methanol as the gain medium when the energy of pump energy is 32.4 mJ at 532 nm. The tunable range is found from 545.3 nm to 573.1 nm for fundamental and 272.5 nm to 286.9 nm after frequency doubling with maximum pulse energy 1.45 mJ at 280 nm, which indicates the frequency doubling efficiency is 18.7%. OH-PLIF images with time interval 0.1 ms are gotten using the dye laser, which proves the feasibility of this dye laser applied in dynamic laser diagnosis fields.

TCSPC technique requires that the signal should be weak with average numbers of photons in one signal period being less than 0.01 to 0.1 in order to ensure the accuracy. If the signals are stronger, the photon pile-up will lead to range walk errors. In this paper, the range walk errors of TCSPC lidar using photomultiplier tube caused by the amplitude change and distortion of reconstructed laser waveform are theoretically analyzed. A correction method for the errors under strong signal situations is proposed and validated in a laser ranging experiment. The results show that, using the correction method, range walk errors can be reduced from 183mm to 6mm for the case when the average number of photons detected is 9.6 and the distance of the target is ∼ 4.8m. The applicable returned signal domain of the TCSPC technique has been extended up to 10 photons per signal period, which has been improved by two orders of magnitude compared to the conventional weak signals.

Time-correlated single-photon counting (TCSPC) is one of the most active technologies for optical time-of-flight ranging and three-dimensional (3D) imaging. Ranging accuracy is one of the most critical issues of lidar system. We propose a three-laser-beam TCSPC lidar system. Besides the main measuring laser beam, the second beam is used to decrease the error caused by jitter of time synchronization. The third beam has n+0.5 time bins’ difference with the main measuring beam, and it is used to correct the error caused by the discrete sampling of time-to-digital converter (TDC). To validate the feasibility of the three-laser-beam TCSPC lidar system, a series of simulation experiments using the scheme is carried out. The results show that this method is effective to improve the ranging accuracy of TCSPC lidar system.

Although spiking neural networks (SNNs) take benefits from the bio-plausible neural modeling, the low accuracy under the common local synaptic plasticity learning rules limits their application in many practical tasks. Recently, an emerging SNN supervised learning algorithm inspired by backpropagation through time (BPTT) from the domain of artificial neural networks (ANNs) has successfully boosted the accuracy of SNNs and helped improve the practicability of SNNs. However, current general-purpose processors suffer from low efficiency when performing BPTT for SNNs due to the ANN-tailored optimization. On the other hand, current neuromorphic chips cannot support BPTT because they mainly adopt local synaptic plasticity rules for simplified implementation. In this work, we propose H2Learn, a novel architecture that can achieve high efficiency for BPTT-based SNN learning which ensures high accuracy of SNNs. At the beginning, we characterized the behaviors of BPTT-based SNN learning. Benefited from the binary spike-based computation in the forward pass and the weight update, we first design lookup table (LUT) based processing elements in Forward Engine and Weight Update Engine to make accumulations implicit and to fuse the computations of multiple input points. Second, benefited from the rich sparsity in the backward pass, we design a dual-sparsity-aware Backward Engine which exploits both input and output sparsity. Finally, we apply a pipeline optimization between different engines to build an end-to-end solution for the BPTT-based SNN learning. Compared with the modern NVIDIA V100 GPU, H2Learn achieves 7.38x area saving, 5.74-10.20x speedup, and 5.25-7.12x energy saving on several benchmark datasets.

Graph convolutional neural networks (GNN) have achieved state-of-the-art performance on tasks like node classification. It has become a new workload family member in data-centers. GNN works on irregular graph-structured data with three distinct phases: Combination, Graph Processing, and Aggregation. While Combination phase has been well supported by sgemm kernels in cuBLAS, the other two phases are still inefficient on GPGPU due to the lack of optimized CUDA kernels. In particular, Aggregation phase introduces large volume of DRAM storage footprint and data movement, and both Aggregation and Graph Processing phases suffer from high kernel launching time. These inefficiencies not only decrease training throughput but also limit users from training GNNs on larger graphs on GPGPU. Although these problems have been partially alleviated by recent studies, their optimizations are still not sufficient. In this paper, we propose fuseGNN, an extension of PyTorch that provides highly optimized APIs and CUDA kernels for GNN. First, two different programming abstractions for Aggregation phase are utilized to handle graphs with different average degrees. Second, dedicated GPGPU kernels are developed for Aggregation and Graph Processing in both forward and backward passes, in which kernel-fusion along with other optimization strategies are applied to reduce kernel launching time and latency as well as exploit data reuse opportunities. Evaluation on multiple benchmarks shows that fuseGNN achieves up to 5.3× end-to-end speedup over state-of-the-art frameworks, and the DRAM storage footprint is reduced by several orders of magnitude on large datasets.

A novel voxel-based spatial elongation filtering method is proposed, to reduce noise in airborne single-photon lidar (SPL) data. In this method, six additional points are generated adjacent to each point of interest in the SPL data. Then, we count the number of points within each voxel and discriminate signals from noise via a predefined threshold. A filter performance evaluation index (taking into account the false alarm and signal loss rates, and the average distance between the residual noise points and their nearest signal points) is introduced. We compare the proposed and voxel-based spatial filtering method. The average false alarm rate found with our method (3.5%) is 18.6% smaller than that of the voxel-based spatial filtering method (4.3%).

Graph convolutional neural networks (GCNs) have achieved state-of-the-art performance on graph-structured data analysis. Like traditional neural networks, training and inference of GCNs are accelerated with GPUs. Therefore, characterizing and understanding the execution pattern of GCNs on GPU is important for both software and hardware optimization. Unfortunately, to the best of our knowledge, there is no detailed characterization effort of GCN workloads on GPU. In this paper, we characterize GCN workloads at inference stage and explore GCN models on NVIDIA V100 GPU. Given the characterization and exploration, we propose several useful guidelines for both software optimization and hardware optimization for the efficient execution of GCNs on GPU., Comment: To Appear in IEEE Computer Architecture Letters

We use a Wiener deconvolution filter to deblur the streak image of streak tube imaging lidar to improve the spatial resolution of the system and reduce the edge blurring effect in point clouds. Experiments were performed to investigate the performance of the deconvolution method. Results show that the spatial resolution improved from 9 to 4.5 mm, and the root-mean-square errors of the edge regions are effectively reduced. Additionally, the transition section decreases from 14 to 5.6 mm when the target is 5 m away from the receiver.

Image sensors based on microchannel plates (MCPs) have important applications as array image sensors owing to their high sensitivity and wide field of view, and because no scanning is required. When MCP-based image sensors, such as ICCD and ICMOS, are used for laser-active imaging, severe diffusion phenomenon occurs due to high electronic gain, and this significantly affects the resulting image quality. This study uses single-photon echo signals to remove the diffusion phenomenon in MCP-based image sensors. A laser-active imaging system based on a MCP image sensor produces single-photon echo signals of the target. The single-photon echo signals are then processed using a weighting algorithm and sub-pixel method, which effectively remove the diffusion phenomenon and reconstruct a high-resolution image of the target. Therefore, when the MCP-based image sensor operates in a high-electronic-gain environment, this method improves the quality of laser-active imaging. The study shows that the method can improve the image resolution obtained by a MCP-based image sensor to 25.51 lp/mm in high-electronic-gain conditions.

This paper describes a full waveform sampling LiDAR system applying stripe principle. A kind of denoising method based on edge detection of original stripe signal is proposed. This method is compared with other denoising methods, such as Wiener filtering, mean filtering and median filtering. It is found that the proposed denoising method is much more effective for dealing with the waveform signals.

A voxel-based spatial elongation filtering method is proposed to remove noise and preserve signal in SPL data. The false alarm rate using our method is 18.6% smaller than that using the voxel-based spatial filtering method. © 2019 The Author(s)

In this paper, the distortion of laser beams in the burst of a burst mode dye laser at 10 kHz repetition rates is researched dynamically. The distortion caused by the thermal lens effects is shown when the repetition rates surpass 2.5 kHz, and is observed emerging 100 μs after the first pulse in the burst. The deflection angles between the laser beams of the first pulse in the burst and each pulse in the burst are measured to describe the distortion. The deflection angles increase to the steady states 400 μs after the first pulse. The maximum deflection angles increase from 0.5 mrad to 0.85 mrad at 10 kHz with the pump pulse energies increasing from 12 mJ to 30 mJ, and increase from 0.23 mrad to 0.4 mrad at 5 kHz with the pump pulse energies increasing from 12 mJ to 30 mJ. The results show that the distortion increases when the pump pulse energies and the repetition rates of the dye laser increase.

Streak tube imaging lidar (STIL) is an active imaging system that has a high range accuracy and a wide range gate with the use of a pulsed laser transmitter and streak tube receiver to produce 3D range images. This work investigates the range accuracy performance of STIL systems based on a peak detection algorithm, taking into account the effects of blurring of the image. A theoretical model of the time-resolved signal distribution, including the static blurring width in addition to the laser pulse width, is presented, resulting in a modified range accuracy analysis. The model indicates that the static blurring width has a significant effect on the range accuracy, which is validated by both the simulation and experimental results. By using the optimal static blurring width, the range accuracies are enhanced in both indoor and outdoor experiments, with a stand-off distance of 10 m and 1700 m, respectively, and corresponding, best range errors of 0.06 m and 0.25 m were achieved in a daylight environment.

Streak tube imaging lidar (STIL) is an active imaging system that has a high range accuracy with the use of a pulsed laser transmitter and streak tube receiver to produce 3D range images. This work investigates the effect of the time bin size on the range accuracy of STIL systems based on the peak detection algorithm. The numerical simulation indicates that the time bin size has a significant effect on the range accuracy, resulting in a modified analytical estimate of the range error. An indoor experiment with a planar target is carried out to validate the theory that shows the linear relationship between the range error and the time bin size. Finer 3D depth images of a fist model are acquired by using a smaller time bin size and the best range error of 0.003 m is achieved with the optimal time bin size of 0.07 ns.

In this study, the Richardson–Lucy (R–L) iteration, Tikhonov (T–K) regularization, and fast iterative shrinkage-thresholding algorithm (FISTA) are first used to deblur streak images. The deblur performances of these three methods are compared with that of Wiener deconvolution. The spatial resolutions of Wiener deconvolution, R–L iteration, T–K regularization, and FISTA improved from 9 mm to 4.5, 5, 5, and 5 mm, respectively. The root-mean-square errors (RMSE) of one -, two -, and three-plane targets decreased effectively using these three deblur methods. The R–L iteration performs the best in terms of the RMSE for the two-plane target. The effects of the parameters used in each method are investigated.

A continuously frequency-tuned laser diode end-pumped Nd:YVO

This article gave a brief research about solid-state dye laser based on PMMA polymer doped with different laser dye through pre-polymerization method, and showed good fluorescence and laser characteristics in experimental research.

Graphitic carbon-doped silica aerogels, including graphite-doped and graphene-doped silica aerogels, were fabricated by the sol–gel method. Thermal resistance was a significant comprehensive parameter which could reflect the ability to prevent the heat transfer. In order to give a direct evaluation of the thermal insulation performance, the calculated formula of thermal resistance for graphitic carbon-doped silica aerogel thermal insulation layers had been given through theoretic analysis on the basis of the thermal conductivity and the aerogel-based heat insulation layers showed good thermal insulation performance. It was expected that theoretical evaluation and experimental research could provide a reference for the reliability and performance in the field of the aerogel-based insulation layers.

Dye-doped distributed feedback (DFB) structure is an essential structure for DFB laser. This paper presents an operative method to design a kind of dye-doped distributed feedback laser based on dye-doped holographic polymer dispersed liquid crystal (HPDLC) matrix. The results show that we have processed DFB structure of 10μm period, with a relatively low period deviation of less than 1%. Furthermore, we gain output characteristics of DFB structure through experimental methods, which show good characteristics for wide tuning range, narrow linewidth laser output production.

Large dynamic range of returned laser signal intensity challenges the lidar waveform sampling (WS) detection schemes. A hybrid lidar system design using the complementarity WS and time correlated single photon counting (TCSPC) has been proposed.