Which Hype for my New Task? Hints and Random Search for Reservoir Computing Hyperparameters

Xavier Hinaut (1) , Nathan Trouvain (1)

1 Mnemosyne - Mnemonic Synergy

In learning systems, hyperparameters are parameters that are not learned but need to be set a priori. In Reservoir Computing, there are several parameters that needs to be set a priori depending on the task. Newcomers to Reservoir Computing cannot have a good intuition on which hyperparameters to tune and how to tune them. For instance, beginners often explore the reservoir sparsity, but in practice this parameter is not of high influence on performance. Most importantly, many authors keep doing suboptimal hyperparameter searches: using grid search as a tool to explore more than two hyperparameters, while restraining the spectral radius to be below unity. In this short paper, we give some suggestions, intuitions, and give a general method to find robust hyperparameters while understanding their influence on performance. We also provide a graphical interface (included in ReservoirPy) in order to make this hyperparameter search more intuitive. Finally, we discuss some potential refinements of the proposed method.

Mots clés: Reservoir Computing; Echo State Networks; Hyperparameters; Random Search; Grid Search; Effective Spectral Radius

Domaines: Neurosciences [q-bio.NC]; Intelligence artificielle [cs.AI]; Réseau de neurones [cs.NE]; Apprentissage [cs.LG]; Linguistique

Xavier Hinaut, Nathan Trouvain. Which Hype for my New Task? Hints and Random Search for Reservoir Computing Hyperparameters. 2021. ⟨hal-03203318v1⟩