Byzantine tolerant gradient descent for distributed machine learning with adversaries
The present application concerns a computer-implemented method for training a machine learning model in a distributed fashion, using Stochastic Gradient Descent, SGD, wherein the method is performed by a first computer in a distributed computing environment and comprises performing a learning round, comprising broadcasting a parameter vector to a plurality of worker computers in the distributed computing environment, receiving an estimate update vector (gradient) from all or a subset of the worker computers, wherein each received estimate vector is either an estimate of a gradient of a cost function, or an erroneous vector, and determining an updated parameter vector for use in a next learning round based only on a subset of the received estimate vectors. The method aggregates the gradients while guaranteeing resilience to up to half workers being compromised (malfunctioning, erroneous or modified by attackers).
60484385
Alternative title(s) : (fr) Descente de gradient tolérant les byzantines pour apprentissage machine distribué avec des adversaires
TTO:6.1813
Patent number | Country code | Kind code | Date issued |
US2020380340 | US | A1 | 2020-12-03 |
WO2019105543 | WO | A1 | 2019-06-06 |