FedHD: Communication-efficient federated learning from hybrid data.

Federated learning (FL) has attracted significant attention in the machine learning community owing to its instinct local privacy awareness. Depending on how the data are distributed over the clients, FL problems can be divided into three categories, namely the horizontal FL (HFL), the vertical FL (VFL) and the hybrid FL (HBFL). Among them, the HBFL problem is the most challenging because each client neither owns the full set of data samples nor knows the complete feature information. While many FL algorithms have been developed for the HFL and VFL problems, unfortunately they cannot handle the HBFL problem. In this paper, we propose a new FL algorithm, termed as FedHD, that can solve the challenging learning task under the HBFL setting. Since the clients cannot perform local optimization on their own under the hybrid data, a tracking variable is introduced to enable the clients to track the global gradient information and update the model based on their local data. FedHD allows the clients to perform multiple steps of local stochastic gradient descent (SGD), and hence has improved communication efficiency. Theoretical analysis is conducted to show that FedHD has a O (1 / Q T) convergence rate, where Q denotes the local update number and T is the total number of communication rounds. Then we further reveal the insights on how various algorithm parameters impact on the convergence performance. Experiment results show that the proposed FedHD exhibits robust performance on the hybrid data, and is largely superior to the naive local training model. [ABSTRACT FROM AUTHOR]