Back to Search
Start Over
FeGAN: Scaling Distributed GANs
- Source :
- ACM/IFIP Middleware 2020-Annual ACM/IFIP Middleware conference, ACM/IFIP Middleware 2020-Annual ACM/IFIP Middleware conference, Dec 2020, Delft / Virtual, Netherlands. ⟨10.1145/3423211.3425688⟩, Middleware, Middleware 2020-ACM/IFIP Middleware, Middleware 2020-ACM/IFIP Middleware, Dec 2020, Virtual, Netherlands. pp.1-14
- Publication Year :
- 2020
- Publisher :
- HAL CCSD, 2020.
-
Abstract
- International audience; Existing approaches to distribute Generative Adversarial Networks (GANs) either (i) fail to scale for they typically put the two components of a GAN (the generator and the discriminator) on different machines, inducing significant communication overhead, or (ii) they face GAN training specific issues, exacerbated by distribution. We propose FeGAN, the first middleware for distributing GANs over hundreds of devices addressing the issues of mode collapse and vanishing gradients. Essentially, we revisit the idea of Federated Learning, co-locating a generator with a discriminator on each device (addressing the scaling problem) and having a server aggregate the devices' models using balanced sampling and Kullback-Leibler (KL) weighting, mitigating training issues and boosting convergence. Through extensive experiments, we show that FeGAN generates high-quality dataset samples in a scalable and devices' heterogeneity tolerant manner. In particular, FeGAN achieves up to 5× throughput gain with 1.5× less bandwidth compared to the state-of-the-art GAN distributed approach (named MD-GAN), while scaling to at least one order of magnitude more devices. We demonstrate that FeGAN boosts training by 2.6× w.r.t. a baseline application of Federated Learning to GANs, while preventing training issues.
- Subjects :
- Generative Adversarial Networks
Discriminator
Boosting (machine learning)
Computer science
Distributed computing
Balanced sampling
02 engineering and technology
010501 environmental sciences
01 natural sciences
Federated learning
Weighting
Machine Learning
ml-ai
[INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG]
[STAT.ML]Statistics [stat]/Machine Learning [stat.ML]
Distributed Systems
Scalability
0202 electrical engineering, electronic engineering, information engineering
020201 artificial intelligence & image processing
[INFO.INFO-DC]Computer Science [cs]/Distributed, Parallel, and Cluster Computing [cs.DC]
Scaling
Federated Learning
Non-iid data
Scaling problem
0105 earth and related environmental sciences
Subjects
Details
- Language :
- English
- Database :
- OpenAIRE
- Journal :
- ACM/IFIP Middleware 2020-Annual ACM/IFIP Middleware conference, ACM/IFIP Middleware 2020-Annual ACM/IFIP Middleware conference, Dec 2020, Delft / Virtual, Netherlands. ⟨10.1145/3423211.3425688⟩, Middleware, Middleware 2020-ACM/IFIP Middleware, Middleware 2020-ACM/IFIP Middleware, Dec 2020, Virtual, Netherlands. pp.1-14
- Accession number :
- edsair.doi.dedup.....2deadf1d78940aaa263d8bd77b03a1fc