How Memories Are Made: Patterns in Neural Space

Where Is a Memory?

Here’s a question that seems simple but gets really interesting when you think about it: Where is a memory stored? Not like where in your head—that’s too easy, it’s in your brain, sure. But where, exactly? Is it in one neuron? That can’t be right—individual neurons die all the time and you don’t suddenly forget your mother’s face. Is it like a file on a hard drive, sitting in some specific location waiting to be read? That’s what it feels like, isn’t it? You “retrieve” a memory, like you’re looking it up in some enormous filing cabinet.

But nature doesn’t work like filing cabinets. The more you look at how memory actually operates, the more you realize it’s not stored in a place at all. It’s stored in a pattern. Specifically, it’s stored in the pattern of connections between neurons—the strengths of the synapses that link them together. When you remember something, you’re not reading a file. You’re running a dynamical process. Your brain is rolling into a valley in a high-dimensional landscape of activity.

Let me explain what I mean by that, because it’s the key to understanding how memory works—both in biological brains and in artificial neural networks.

Think about a ball on a hilly surface. If you place it on a slope, it rolls downhill until it settles in a valley. The valley is stable: push the ball a little and it rolls back. That’s what we call an attractor—a state the system naturally falls into. Now, what if each valley represents a different memory? You give your brain a partial cue—maybe you hear a few notes of a song—and that initializes your neural activity somewhere on the hillside. The dynamics of your neural network then take over, and your activity rolls downhill into the nearest valley, reconstructing the full memory of that song. You’re not searching through a database. You’re following the geometry of an energy landscape.

This is exactly how associative memory works. The landscape is carved by the pattern of synaptic weights—the connections between neurons. Memories are valleys, and remembering is the process of falling into one.

Valleys in Neural Activity Space

Let’s get more concrete. Imagine a network of neurons, each one either firing or silent at any given moment. The state of the network at any instant is just a list of which neurons are on and which are off. If you have, say, a hundred neurons, then the state is a point in a hundred-dimensional space. I know, I know—hundred-dimensional space sounds scary. But it’s just a way of keeping track of all the neurons at once. Each neuron is a coordinate, and the whole pattern of activity is a point.

Now, as time passes, that point moves. The network’s activity traces out a trajectory through this high-dimensional state space. Different behaviors—different thoughts, different memories—correspond to different trajectories. And here’s the beautiful thing: the network’s wiring constrains which trajectories are possible. The connections between neurons create a landscape, and the activity flows along that landscape like water flowing downhill.

This isn’t some abstract metaphor. When you record from populations of neurons in a monkey’s motor cortex while it’s moving its arm, you can literally watch this happen. Ninety neurons firing at different rates give you a point in ninety-dimensional space, and as the monkey plans and executes a movement, that point traces a smooth path. Different movements trace different paths, and those paths aren’t arbitrary—they’re shaped by the underlying circuitry. The monkey can’t just make its neurons do anything. The anatomy carves out a manifold, a lower-dimensional surface embedded in the high-dimensional space, and the activity is constrained to flow along that surface.

The same geometric picture applies to memory. When you store a memory, you’re carving a valley into the activity landscape. When you recall it, you’re letting the dynamics flow into that valley. The question is: how do you carve the valley in the first place?

The answer turns out to be remarkably simple. It’s a principle called Hebbian learning, and you’ve probably heard it summarized as “neurons that fire together, wire together.” If two neurons are active at the same time, strengthen the connection between them. If they’re anti-correlated—one fires when the other is silent—weaken the connection. That’s it. It’s a purely local rule: each synapse only needs to know about the two neurons it connects.

But look what this does. Suppose you want to store a specific pattern of activity—some neurons on, some off, representing a particular memory. For every pair of neurons that are both active in that pattern, you increase the weight of the connection between them. For pairs where one is active and the other isn’t, you decrease the weight. Now, when you initialize the network with a noisy or partial version of that pattern, the dynamics push you toward the full pattern. The neurons that fire together in the target memory reinforce each other, pulling the network’s activity toward the stored state. You’ve created a valley, and the network naturally falls into it.

If you want to store multiple memories, you just add them up. For each memory pattern, compute the weight changes using the Hebbian rule, and sum all the contributions. Each memory carves its own valley. When you initialize the network somewhere in the landscape, it rolls into whichever valley is closest. That’s content-addressable memory: you don’t need to know the address of the memory you’re looking for. You give a hint—a fragment of a song, a blurry face—and the dynamics do the rest.

This is how Hopfield networks work. They’re simple models, but they capture something profound: memory as energy minimization. The network has an energy function, and the dynamics always move toward lower energy. The memories are local minima. Retrieval is just falling downhill.

Building Memories in Layers

Now, here’s where it gets even more interesting. Real memories aren’t just single patterns. They’re hierarchical. Your memory of your grandmother’s face isn’t a direct pattern of light hitting your retina. It’s built from edges, which combine into corners and textures, which combine into facial features, which combine into the recognizable gestalt of a face. This hierarchical structure is baked into the architecture of both biological brains and artificial neural networks.

In deep neural networks—like the ones that revolutionized computer vision—you see this hierarchy emerge automatically during learning. The first layer learns to detect simple edges and color blobs. The second layer combines those edge detectors into corners and simple shapes. By the fifth layer, you’ve got neurons that respond to faces, even though the network was never explicitly told what a face is. The hierarchy builds itself through the learning process, with each layer constructing more abstract features by combining the outputs of the previous layer.

This is spectacularly efficient. You don’t need to learn a separate face detector from scratch. You reuse the edge detectors and corner detectors you’ve already learned. With just a few layers, you can represent exponentially more complex patterns than you could with a single layer. In fact, a deep network with a hundred and thirty neurons can learn patterns that a shallow network with a hundred thousand neurons can’t capture. That’s not a typo. Depth multiplies your representational capacity by allowing composition. Instead of every neuron working independently, later neurons operate on the complex features created by earlier ones.

The brain does the same thing. Visual cortex has a hierarchy: V1 responds to edges, V2 to more complex shapes, V4 to objects, and so on. Each level builds on the previous one, extracting increasingly abstract structure from the raw sensory data. Memory formation follows this hierarchy. When you encode an experience, you’re not storing pixels. You’re storing a pattern of activity across this multilayered representational space, a pattern that reflects the hierarchical structure of what you’re remembering.

And here’s something fascinating: memories aren’t independent. When you form two memories close together in time, they share neurons. There’s a time window—a few hours, in experiments with mice—during which neurons that participated in the first memory are more likely to be recruited into the second. Their excitability is elevated. The two memories overlap, sharing part of their neural substrate. This creates a link: if you extinguish one memory, you partially extinguish the other. But if you wait a day, the excitability has decayed, and the new memory recruits a different population. The memories become independent.

This temporal linking gives you a mechanism for building associative structures—chains and clusters of related memories—out of overlapping neural patterns. It’s another kind of geometry, this time in the space of memory traces rather than the space of real-time activity.

There’s one more twist. Your brain doesn’t treat all information equally. It prioritizes things that are relevant to you. When you hear your own name in a noisy room, it immediately grabs your attention. When you encounter information that connects to your self-concept, you encode it more deeply. This self-reference effect is so powerful that it shapes what you remember and what you forget. Your memories aren’t an objective record. They’re filtered through the lens of personal relevance, biased toward things that matter to your survival and identity.

In other words, the valleys in your memory landscape aren’t evenly distributed. They’re clustered around the things that concern you. Evolution built you to pay attention to what’s important to you, and that bias sculpts the structure of your memories.

Remembering as Dynamics

So what does all this add up to? Memory isn’t storage. It’s dynamics. Remembering isn’t retrieval. It’s reconstruction. You don’t have a library of experiences filed away somewhere. You have a network of synapses that defines a landscape of possible activity patterns, and memory is the process of flowing through that landscape toward stable states.

This changes how we think about memory in a fundamental way. It explains why memories are malleable—every time you remember something, you’re reconstructing it, and the reconstruction can be influenced by context, by what you expect, by how you feel. You’re not playing back a recording. You’re running a dynamical system, and dynamical systems are sensitive to their initial conditions and their environment.

It also explains why memory is so robust. If a few neurons die, no big deal—the valley is still there, shaped by the remaining connections. If you get only a fragment of a cue, the dynamics still pull you toward the full memory, as long as the fragment is close enough to the valley. This is content-addressable recall: you don’t need the exact right key. Anything in the neighborhood works.

And it explains the efficiency of hierarchical representations. You don’t need to store every possible combination of features. You store the building blocks—edges, textures, shapes—and you compose them on the fly. This is why deep networks work so well, and it’s probably why brains are organized in layers.

The geometry of memory reveals something beautiful: the brain and artificial neural networks aren’t doing magic. They’re doing physics. They’re minimizing energy, following gradients, settling into attractors. Memory is patterns in synaptic weights. Remembering is rolling into valleys. Learning is carving those valleys by strengthening connections between co-active neurons.

It’s simple, really. Neurons that fire together, wire together. That local rule, applied over and over, builds a landscape of memory. And when you want to remember something, you just let your activity roll downhill until you land in the right valley. No filing cabinet. No database search. Just geometry.

And that, to me, is more elegant than any filing cabinet could ever be.