Tracking Performance of Online Stochastic Learners

The utilization of online stochastic algorithms is popular in large-scale learning settings due to their ability to compute updates on the fly, without the need to store and process data in large batches. When a constant step-size is used, these algorithms also have the ability to adapt to drifts in problem parameters, such as data or model properties, and track the optimal solution with reasonable accuracy. Building on analogies with the study of adaptive filters, we establish a link between steady-state performance derived under stationarity assumptions and the tracking performance of online learners under random walk models. The link allows us to infer the tracking performance from steady-state expressions directly and almost by inspection.


Published in:
Ieee Signal Processing Letters, 27, 1385-1389
Year:
Jan 01 2020
Publisher:
Piscataway, IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
ISSN:
1070-9908
1558-2361
Keywords:




 Record created 2020-09-06, last modified 2020-10-26


Rate this document:

Rate this document:
1
2
3
 
(Not yet reviewed)