Search

Your search keyword '"Rajawat, Ketan"' showing total 23 results
Start Over Author "Rajawat, Ketan" Journal ieee transactions on signal processing
23 results on '"Rajawat, Ketan"'

4. Optimized Gradient Tracking for Decentralized Online Learning

Author

Sharma, Shivangi Dubey and Rajawat, Ketan

Abstract

This work considers the problem of decentralized online learning, where the goal is to track the optimum of the sum of time-varying functions, distributed across several nodes in a network. The local availability of the functions and their gradients necessitates coordination and consensus among the nodes. We put forth the Generalized Gradient Tracking (GGT) framework that unifies a number of existing approaches, including the state-of-the-art ones. The performance of the proposed GGT algorithm is theoretically analyzed using a novel semidefinite programming-based analysis that yields the desired regret bounds under very general conditions and without requiring the gradient boundedness assumption. The results are applicable to the special cases of GGT, which include various state-of-the-art algorithms as well as new dynamic versions of various classical decentralized algorithms. To further minimize the regret, we consider a condensed version of GGT with only four free parameters. A procedure for offline tuning of these parameters using only the problem parameters is also detailed. The resulting optimized GGT (oGGT) algorithm not only achieves improved dynamic regret bounds, but also outperforms all state-of-the-art algorithms on both synthetic and real-world datasets.

Published
2024
Full Text
View/download PDF

13. Escaping Saddle Points for Successive Convex Approximation.

Author

Bedi, Amrit Singh, Rajawat, Ketan, Aggarwal, Vaneet, and Koppel, Alec

Subjects
*MULTIDIMENSIONAL scaling, *PATTERN recognition systems, *RANDOM forest algorithms, *MACHINE learning, *APPROXIMATION algorithms
Abstract

Optimizing non-convex functions is of primary importance in modern pattern recognition because it underlies the training of deep networks and nonlinear dimensionality reduction. First-order algorithms under suitable randomized perturbations or step-size rules have been shown to be effective for such settings as their limit points can be guaranteed to be local extrema rather than saddle points. However, it is well-known that the practical convergence of first-order methods is slower than those which exploit additional structure. In particular, empirically, successive convex approximation (SCA) converges faster than first-order methods. However, to date, SCA in general non-convex settings converges to first-order stationary points, which could either be local extrema or saddle points whose performance is typically inferior. To mitigate this issue, we propose calibrated randomized perturbations of SCA, which exhibit the improved convergence rate as compared to the gradient descent counter part. In particular, our main technical contributions are to establish the non-asymptotic performance of SCA algorithm and its perturbed variant converges to an approximate second-order stationary point. Experiments on multi-dimensional scaling, a machine learning problem whose training objective is non-convex, substantiate the performance gains associated with employing random perturbations. [ABSTRACT FROM AUTHOR]

Published
2022
Full Text
View/download PDF

Catalog

Books, media, physical & digital resources
See catalog results