Iterative Learning and Denoising in Convolutional Neural Associative Memories

The task of a neural associative memory is to retrieve a set of previously memorized pat- terns from their noisy versions by using a net- work of neurons. Hence, an ideal network should be able to 1) gradually learn a set of patterns, 2) retrieve the correct pattern from noisy queries and 3) maximize the number of memorized patterns while maintaining the reliability in responding to queries. We show that by considering the inherent redundancy in the memorized patterns, one can obtain all the mentioned properties at once. This is in sharp contrast with previous work that could only improve one or two aspects at the expense of the others. More specifically, we devise an iterative algorithm that learns the redundancy among the patterns. The resulting network has a retrieval capacity that is exponential in the size of the network. Lastly, by considering the local structures of the net- work, the asymptotic error correction performance can be made linear in the size of the network.

Published in:
Proceedings 30th International Conference on Machine Learning (ICML), 28, 1, 445-453
Presented at:
30th International Conference on Machine Learning (ICML), Atlanta, USA, June 16-21, 2013

 Record created 2013-03-25, last modified 2018-03-18

Publisher's version:
Download fulltextPDF
External link:
Download fulltextURL
Rate this document:

Rate this document:
(Not yet reviewed)