Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Journal articles
  4. Nonbinary Associative Memory With Exponential Pattern Retrieval Capacity and Iterative Learning
 
research article

Nonbinary Associative Memory With Exponential Pattern Retrieval Capacity and Iterative Learning

Salavati, Amir Hesam  
•
Kumar, K. Raj
•
Shokrollahi, Amin  
2014
Ieee Transactions On Neural Networks And Learning Systems

We consider the problem of neural association for a network of nonbinary neurons. Here, the task is to first memorize a set of patterns using a network of neurons whose states assume values from a finite number of integer levels. Later, the same network should be able to recall the previously memorized patterns from their noisy versions. Prior work in this area consider storing a finite number of purely random patterns, and have shown that the pattern retrieval capacities (maximum number of patterns that can be memorized) scale only linearly with the number of neurons in the network. In our formulation of the problem, we concentrate on exploiting redundancy and internal structure of the patterns to improve the pattern retrieval capacity. Our first result shows that if the given patterns have a suitable linear-algebraic structure, i.e., comprise a subspace of the set of all possible patterns, then the pattern retrieval capacity is exponential in terms of the number of neurons. The second result extends the previous finding to cases where the patterns have weak minor components, i.e., the smallest eigenvalues of the correlation matrix tend toward zero. We will use these minor components (or the basis vectors of the pattern null space) to increase both the pattern retrieval capacity and error correction capabilities. An iterative algorithm is proposed for the learning phase, and two simple algorithms are presented for the recall phase. Using analytical methods and simulations, we show that the proposed methods can tolerate a fair amount of errors in the input while being able to memorize an exponentially large number of patterns.

  • Files
  • Details
  • Metrics
Type
research article
DOI
10.1109/Tnnls.2013.2277608
Web of Science ID

WOS:000331985500010

Author(s)
Salavati, Amir Hesam  
Kumar, K. Raj
Shokrollahi, Amin  
Date Issued

2014

Publisher

Ieee-Inst Electrical Electronics Engineers Inc

Published in
Ieee Transactions On Neural Networks And Learning Systems
Volume

25

Issue

3

Start page

557

End page

570

Subjects

Dual-space method

•

error correcting codes

•

message passing

•

neural associative memory

•

stochastic learning

•

algoweb_bio

Editorial or Peer reviewed

REVIEWED

Written at

EPFL

EPFL units
ALGO  
Available on Infoscience
April 2, 2014
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/102366
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés