Increasing Capacity of Association Memory by Means of Synaptic Clustering
Résumé
Making association is an essential property of human cognition. Systems that try to mimic this process and to make a coherent model of the world should have robust and high capacity association memory. Findings of nonlinear properties of dendritic tree suggest an alternative way how neurons can store associations. In this paper, we present a minimalistic neuron model with clustered synapses and show that it provides much higher association memory capacity compared to traditional models. Due to properties of sparse activation and tracking higher-order correlations in the input pattern an individual neuron can recognize thousands of patterns. Theoretical examination shows that this high capacity is reached because learning exact combinations of active neurons extends the dimension of an input space and thus increases pattern separability. We argue that such beneficial computational properties is realized in biological neural networks through synaptic clustering and sustaining sparse activity in memory-related areas.