A balanced memory network

September 7, 2007

Ever wonder how much information we put in our heads? The answer: a lot. For starters, a typical vocabulary is 50,000-250,000 words. And then there are all the little details that stretch back decades – the house we grew up in, the time we spilled orange juice on our test back in third grade, the solution to a quadratic equation (for some of us).

So where do we put it all? If we had hard drives in our heads, the answer would be easy: we would store memories as 0s and 1s. But we don't, we have neurons, connected by synapses, and storing memories in such systems is a lot harder than putting 0s and 1s on a hard drive.

Nevertheless, about two decades ago John Hopfield showed that memories could be stored by modifying the strength of synapses in a particular way. Importantly, the number of memories that could be stored using his scheme was proportional to the number of neurons in the network. This solved the storage problem: there are about 50 million neurons in a cubic centimeter of cortex, plenty of room for both a vocabulary and spilled orange juice.

Recently, Roudi and Latham from University College London threw a monkey wrench into this picture. In a study publishing in PLoS Computational Biology on September 7, 2007, they show that for realistic networks of spiking neurons, the number of memories is not proportional to the number of neurons, it's proportional to the number of connections per neurons -- at most about 10,000. Moreover, they provided evidence that the constant of proportionality is small, not more than a few percent, and they eliminated one of theorists' favorite tricks -- reducing the number of neurons involved in any one memory -- for increasing that constant. Thus, if networks use the algorithm proposed by Hopfield, they can store at most about 500 memories, no matter how many neurons they contain.

So we're not exactly back to square one, but we're not much farther than square two: we no longer know how the brain holds so many memories. Roudi and Latham speculate that the answer lies in multiple, weakly coupled networks. However, until that, or some other idea, is shown to be correct, we will have to be content with just remembering, without the added knowledge of how we remember.

Source: Public Library of Science