Back to Search Start Over

Communication Efficient Asynchronous ADMM for General Form Consensus Optimization

Authors :
WANG Dong-xia, LEI Yong-mei, ZHANG Ze-yu
Source :
Jisuanji kexue, Vol 49, Iss 11, Pp 309-315 (2022)
Publication Year :
2022
Publisher :
Editorial office of Computer Science, 2022.

Abstract

The distributed alternating direction method of multipliers(ADMM) is one of the most widely used methods for solving large-scale machine learning applications.However,most distributed ADMM algorithms are based on full model updates.With the increasing of system scale and data volume,the communication cost has become the bottleneck for the distributed ADMM when big data are involved.In order to reduce the communication cost in a distributed environment,a general form consensus asynchronous distributed alternating direction method of multipliers(GFC-ADADMM) is proposed in this paper.First,in the GFC-ADADMM,the associated model parameters rather than full model parameters are transmitted among nodes to reduce the transmission load,and the associated model parameters are filtered according to the characteristics of high-dimensional sparse data sets to further reduce the transmission load.Second,the GFC-ADMM is implemented by an asynchronous allreduce framework,which combines the advantage of the asynchronous communication protocol and the allreduce communication mode.Third,combining the advantages of the stale synchronous parallel(SSP) computing model,allreduce communication model,and hybrid programming model,the asynchronous allreduce framework and MPI/OpenMP hybrid programming model are adopted to implement the GFC-ADADMM,which improves calculation efficiency and communication efficiency of the algorithm.Finally,the sparse logistic regression problem is solved by the GFC-ADADMM.Evaluation with large-scale datasets shows that compared with state-of-the-art asynchronous distributed ADMM algorithms,the GFC-ADADMM can reduce the total running time by 15%-63%,and has higher accuracy in convergence.

Details

Language :
Chinese
ISSN :
1002137X
Volume :
49
Issue :
11
Database :
Directory of Open Access Journals
Journal :
Jisuanji kexue
Publication Type :
Academic Journal
Accession number :
edsdoj.46050bf7803e4eeeba746e526772457c
Document Type :
article
Full Text :
https://doi.org/10.11896/jsjkx.211200006