Search

Pruning the weights of neural networks is an effective and widely-used technique for reducing model size and inference complexity. We develop and test a novel method based on compressed sensing which combines the pruning and training into a single step. Specifically, we utilize an adaptively weighted $\ell^1$ penalty on the weights during training, which we combine with a generalization of the regularized dual averaging (RDA) algorithm in order to train sparse neural networks. The adaptive weighting we introduce corresponds to a novel regularizer based on the logarithm of the absolute value of the weights. We perform a series of ablation studies demonstrating the improvement provided by the adaptive weighting and generalized RDA algorithm. Furthermore, numerical experiments on the CIFAR-10, CIFAR-100, and ImageNet datasets demonstrate that our method 1) trains sparser, more accurate networks than existing state-of-the-art methods; 2) can be used to train sparse networks from scratch, i.e. from a random initialization, as opposed to initializing with a well-trained base model; 3) acts as an effective regularizer, improving generalization accuracy.

Pulse feeling, representing the tactile arterial palpation of the heartbeat, has been widely used in traditional Chinese medicine (TCM) to diagnose various diseases. The quantitative relationship between the pulse wave and health conditions however has not been investigated in modern medicine. In this paper, we explored the correlation between pulse pressure wave (PPW), rather than the pulse key features in TCM, and pregnancy by using deep learning technology. This computational approach shows that the accuracy of pregnancy detection by the PPW is 84% with an AUC of 91%. Our study is a proof of concept of pulse diagnosis and will also motivate further sophisticated investigations on pulse waves.

Bridging equilibrium and nonequilibrium statistical physics attracts sustained interest. Hallmarks of nonequilibrium systems include a breakdown of detailed balance, and an absence of a priori potential function corresponding to the Boltzmann-Gibbs distribution, without which classical equilibrium thermodynamical quantities could not be defined. Here, we construct dynamically the potential function through decomposing the system into a dissipative part and a conservative part, and develop a nonequilibrium theory by defining thermodynamical quantities based on the potential function. Concepts for equilibrium can thus be naturally extended to nonequilibrium steady state. We elucidate this procedure explicitly in a class of time-dependent linear diffusive systems without mathematical ambiguity. We further obtain the exact work distribution for an arbitrary control parameter, and work equalities connecting nonequilibrium steady states. Our results provide a direct generalization on Jarzynski equality and Crooks fluctuation theorem to systems without detailed balance., Comment: 7 pages