Wang, Chengcheng
Zhang, Yonggang
Ying, Bicheng
Sayed, Ali H.
Coordinate-Descent Diffusion Learning by Networked Agents
IEEE Transactions on Signal Processing
1941-0476
10.1109/TSP.2017.2757903
66
2
352-367
16
This paper examines the mean-square error performance of diffusion stochastic algorithms under a generalized coordinate-descent scheme. In this setting, the adaptation step by each agent is limited to a random subset of the coordinates of its stochastic gradient vector. The selection of coordinates varies randomly from iteration to iteration and from agent to agent across the network. Such schemes are useful in reducing computational complexity at each iteration in power-intensive large data applications. They are also useful in modeling situations where some partial gradient information may be missing at random. Interestingly, the results show that the steady-state performance of the learning strategy is not always degraded, while the convergence rate suffers some degradation. The results provide yet another indication of the resilience and robustness of adaptive distributed strategies.
Coordinate descent;
stochastic partial update;
computational complexity;
diffusion strategies;
stochastic gradient algorithms;
strongly-convex cost;
Ieee-Inst Electrical Electronics Engineers Inc
Piscataway
2018
https://arxiv.org/abs/1607.01838;