Search

To solve the problem of low accuracy in the traditional FunkSVD recommendation algorithm, an improved FunkSVD algorithm (RM-FS) is proposed. RM-FS is an improvement of the traditional FunkSVD algorithm, using RMSProp, a deep learning optimization algorithm. The RM-FS algorithm can not only solve the problem of reduced accuracy of the traditional FunkSVD algorithm because of iterative oscillations but also alleviate the impact of data sparseness on the accuracy of the algorithm, achieving the effect of improving the accuracy of the traditional algorithm. The experimental results show that the RM-FS algorithm proposed in this paper effectively improves the accuracy of the recommendation algorithm, which is better than the traditional FunkSVD recommendation algorithm and other improved FunkSVD algorithms. [ABSTRACT FROM AUTHOR]

Nowadays, large and complex deep learning (DL) models are increasingly trained in a distributed manner across multiple worker machines, in which extensive communications between workers pose serious scaling problems. In this article, we present a quantitative survey of communication optimization techniques for data parallel distributed DL. We first identify the major communication challenges and classify the existing solutions into three levels, namely the learning algorithm, the system architecture, and the network infrastructure. We present the state-of-the-art communication optimization techniques and conduct a comparative study of seven common lossless distributed DL methods on a 32-GPU cluster with 100Gb/s InfiniBand (IB). We show that the DL models with low model intensity (such as BERT and BERT-Large) are difficult to scale out even with the best available lossless algorithm over 100Gb/s IB; and the system architecture and scheduling algorithms have a critical impact on the scaling property. We conclude the article with discussions of open issues for further investigation. [ABSTRACT FROM AUTHOR]