Reservoir optimization in recurrent neural networks using properties of Kronecker product

Recurrent neural networks based on reservoir computing are increasingly being used in many applications. Optimization of the topological structure of the reservoir and the internal connection weights for a given task is one of the most important problems in reservoir computing. In this paper, considering the fact that one can construct a large matrix using Kronecker products of several small-size matrices, we propose a method to optimize the reservoir. Having a small number of parameters to optimize, a gradient based algorithm is applied to optimize parameters, and consequently the reservoir. In addition to reducing the number of parameters for optimization, potentially, the method is able to control several other properties of the reservoir such as spectral radius, sparsity, weight distribution and underlying connections, i.e. connection topology. To reveal the effectiveness of the proposed optimization method, the application to the following tasks are considered: Nonlinear autoregressive moving average and multiple superimposed oscillators. Simulation results show satisfactory performance of the method.

Published in:
Logic Journal of IGPL, 18, 5, 670-685
First published online: September 3, 2009 This article appears in: Special Issue: Challenges for recurrent neural networks

 Record created 2009-09-16, last modified 2018-03-17

External link:
Download fulltext
Rate this document:

Rate this document:
(Not yet reviewed)