The impact of Phase Change Memory (PCM) as well as other Non-Volatile Memory (NVM) device characteristics on the quantitative classification performance of artificial neural networks is studied. Our results show that any NVM-based neural network — not just those based on PCM — can be expected to be highly resilient to random effects (device variability, yield, and stochasticity), but will be highly sensitive to “gradient” effects that act to steer all synaptic weights. Asymmetry, such as that found with PCM, can be mitigated by an occasional RESET strategy, which can be both infrequent and inaccurate. Algorithms that can finesse some of the imperfections of NVM devices are proposed.