Background. In medicine and other applications, the copying and sharing of data is impractical for a range of well-considered reasons. With federated learning (FL) techniques, machine learning models can be trained on data spread across several locations without such copying and sharing. While good privacy guarantees can often be made, FL does not automatically incentivize participation and the resulting model can suffer if data is non-identically distributed (non-IID) across locations. Model personalization is a way of addressing these concerns. Methods. In this study, we introduce Weight Erosion: an SGD-based gradient aggregation scheme for personalized collaborative ML. We evaluate this scheme on a binary classification task in the Titanic data set. Findings. We demonstrate that the novel Weight Erosion scheme can outperform two baseline FL aggregation schemes on a classification task, and is more resistant to over-fitting and non-IID data sets.