Search

The size of deep neural networks (DNNs) grows rapidly as the complexity of the machine learning algorithm increases. Distributed deep learning based on model parallelism has been widely used to satisfy the requirements of DNN training related to computation and memory. In this paper, we propose a training framework for pipeline parallelism called BaPipe (Balanced Pipeline) that can automatically explore methods to schedule pipeline parallelism and balanced partition strategies for DNN training on heterogeneous accelerator clusters. In BaPipe, each accelerator calculates the forward and backward propagation for the assigned partition of networks to implement an intra-batch pipeline parallelism strategy. By considering the parameters of DNN models as well as the computation, memory, and communication resources of each accelerator, BaPipe automatically selects the most suitable method of pipeline scheduling from among multiple proposed scheduling modes. It also uses a novel strategy to automatically investigate load balancing in the context of inter-layer partition, intra-layer partition, and coarse-grained partition. We trained such DNNs as VGG-16, ResNet-50, and Google's Neural Machine Translation (GNMT) on GPU clusters, and simulated the training-related performance of FPGA clusters. Compared with the state-of-the-art frameworks for data parallelism (DP) and pipeline parallelism, BaPipe provides a speedup of 3. 2 × and 4 × of memory reduction on various homogeneous and heterogeneous platforms. [ABSTRACT FROM AUTHOR]

In this paper, a wide input range supply voltage tolerant capacitive sensor readout circuit using on-chip solar cell is presented. Based on capacitance controlled oscillators (CCOs) for ultra-low voltage/power consumption, the sensor readout circuit is directly powered by the on-chip solar cell to improve the overall system energy efficiency. An extended sensing range with high sensing accuracy is achieved using a two-step successive approximation register (SAR) and delta-sigma () analog-to-digital (A/D) conversion (ADC) scheme. Digital controls are generated on-chip using a customized sub-threshold digital standard cell library. Systematic error analysis and optimization including the finite switch on-resistance, buffer input-dependent delay, and SAR quantization nonlinearity are also outlined. High power supply rejection ratio (PSRR) is ensured by using a pseudo-differential topology with ratiometric readout. The complete sensing system is implemented using a standard 0.18m complementary metal-oxide-semiconductor (CMOS) process. Simulation results show that the readout circuit achieves a wide input range from 1.5pF to 6.5pF with a worst case PSRR of 0.5% from 0.3V to 0.42V (0.67% from 0.3V to 0.6V). With a 3.5pF input capacitance and a 0.3V supply, the stage achieves a resolution of 7.1-bit (corresponding to a capacitance of 2.2fF/LSB) with a conversion frequency of 371Hz. With an average power consumption of 40nW and a sampling frequency of 47.5kHz, a figure-of-merit (FoM) of 0.78pJ/conv-step is achieved. [ABSTRACT FROM AUTHOR]