Search

Since its release, the Tensorflow framework has been widely used in various fields due to its advantages in deep learning. However, it is still at its early state. Its native distributed implementation has difficulty in expanding for large models because it has issues of low utilization of multiple GPUs and slow distribution compared with running on single machine. It is of great significance to reduce the training time through parallel models. In view of this, we firstly provided an in-depth analysis of the implementation principle of Tensorflow and identify the bottlenecks of its native distributed parallel models to improve. Then, two optimal algorithms are designed and implemented based on data parallelism and model parallelism modes of Tensorflow. For data parallelism, the proposed algorithm is implemented to replace the native linear execution mode with pipeline execution mode. As for model parallelism, the native random partitioning mode is replaced by our proposed novel greedy algorithm. Finally, we built a homogeneous distributed cluster and a heterogeneous distributed cluster respectively to verify the effectiveness of the proposed algorithms. Through a number of comparative experiments, we showed that the proposed optimal parallel algorithms can effectively reduce model training time by an average of 26.5%(or average 1.5x speedup than native distributed algorithms) and improve the utilization of the cluster while keeping the same accuracy level of native Tensorflow. [ABSTRACT FROM AUTHOR]

Grammatical Error Correction (GEC) is the task of correcting several diverse errors in a text such as spelling, punctuation, morphological, and word choice typos or mistakes. Expressed as a sentence correction task, models such as neural-based sequence-to-sequence (seq2seq) GECs have emerged to offer solutions to the task. However, neural-based seq2seq grammatical error correction models are computationally expensive both in training and in translation inference. Also, they tend to suffer from poor generalization and arrive at inept capabilities due to limited error-corrected data, and thus, incapable of effectively correcting grammar. In this work, we propose the use of Neural Cascading Architecture and different techniques in enhancing the effectiveness of neural sequence-to-sequence grammatical error correction models as inspired by post-editing processes of Neural Machine Translations (NMTs). The findings of our experiments show that, in low-resource NMT models, adapting the presented cascading techniques unleashes performances that is comparable to high setting NMT models, with improvements on state-of-the-art (SOTA) JHU FLuency- Extended GUG corpus (JFLEG) parallel corpus for developing and evaluating GEC model systems. We extensively exploit and evaluate multiple cascading learning strategies and establish best practices toward improving neural seq2seq GECs. [ABSTRACT FROM AUTHOR]

3D-R2N2 and other advanced 3D reconstruction neural networks have achieved impressive results, however most of them still suffer from training difficulties and detail losing, due to their weak feature extraction capability and improper loss function. This paper aims to overcome these shortcomings and defects by building a brand new model based on 3D-R2N2. The new model adopts densely connected structure as encoder, and utilizes Chamfer Distance as loss function. The aim is to enhance the learning ability of the network for complex data, meanwhile, make the focus of the whole network rest on the reconstruction of detail structures. In addition, we also made an improved decoder by building two parallel predictor branches to make better use of the feature information and boost the network's performance on reconstruction task. Through extensive tests, the results show that our proposed model called 3D-R2N2-V2 is slightly slower than 3D-R2N2 in predicting speed, but it can be 20% to 30% faster than 3D-R2N2 in training speed and obtain 15% and 10% better voxel IoU results on both single- and multi-view reconstruction tasks, respectively. Compared with other recent state-of-the-art methods like OGN and DRC, the reconstruction effect of our approach is also competitive. [ABSTRACT FROM AUTHOR]