Half-Lives: Radioactive Decay and Information Degradation

In my laboratory, radium-226 decays with mathematical precision—half the sample transforming every 1,600 years, releasing radiation as unstable nuclei seek stability. The process follows exponential decay: predictable in aggregate, random for individual atoms. No external force triggers this transformation; nuclear instability alone drives the emission. What strikes me now is how this pattern appears across utterly different substrates: in human memory, in artificial neural networks, even in the propagation of gradients through computational layers. Is information itself inherently unstable, requiring constant maintenance against inevitable degradation?

Exponential Decay: Radioactivity, Memory, and Networks

The mathematics are identical across domains. Radioactive decay proceeds exponentially because quantum tunneling probability remains constant per unit time—each atom has the same chance of decay regardless of how long it has existed. Human memory follows the same curve: Ebbinghaus demonstrated exponential forgetting, with half the learned material lost within days, then slower subsequent decay. Even our most vivid recollections—flashbulb memories of significant events—degrade over time despite feeling photographically permanent. The subjective certainty persists while the actual content shifts.

Neural networks exhibit parallel phenomena. When learning new tasks, networks experience catastrophic forgetting—new gradient updates overwrite previously learned weight patterns exponentially. Dead neurons emerge when ReLU activation functions zero out entire regions; once a neuron outputs zero across all inputs, gradients through that region also vanish. Without gradient information, backpropagation cannot recover the neuron’s contribution. It dies permanently, wasting model capacity just as unstable isotopes waste nuclear potential through spontaneous emission.

The pattern suggests something fundamental: information stored in any physical substrate faces natural decay pressures. Why should this be universal?

Spontaneous vs. Induced Degradation

The mechanisms differ in crucial ways. Radium decays spontaneously—no external trigger required, only internal nuclear instability. Similarly, gradients vanishing as they propagate through deep network layers represents spontaneous information loss, independent of new learning. The signal degrades layer by layer, like radiation intensity diminishing with distance from source.

But memory reconstruction operates differently. Each recall is an opportunity for mutation. The brain reassembles distributed memory fragments, editing for coherence, filling gaps with invented context. Every access event alters the trace—induced degradation through interference rather than spontaneous decay. Neural networks show both patterns: catastrophic forgetting requires interference from new learning, while gradient vanishing happens spontaneously through architectural constraints.

This distinction matters. Against spontaneous decay we are nearly powerless—my notebooks remain radioactive a century later, decay products persisting despite our wishes. But induced degradation from interference might be preventable through better design: careful learning rates, alternative activation functions like Leaky ReLU that preserve small gradients, architectural innovations that protect old knowledge while acquiring new.

Residual Radiation: Traces That Persist

Yet nothing decays completely to zero. Radium transforms through a decay series—radon, polonium, lead—each daughter product carrying information about its progenitor, if indirectly. Human memories leave emotional residues even when factual content vanishes entirely; the fear response persists though the triggering event grows unclear. Neural networks retain subtle biases after catastrophic forgetting—weight manifold structure, meta-learning patterns that influence future acquisition.

Perhaps information approaches an asymptotic minimum rather than absolute zero. The question becomes: what determines each system’s half-life? For isotopes, nuclear forces. For synaptic memories, plasticity mechanisms and interference patterns. For artificial networks, architecture choices and optimization constraints. We cannot prevent all decay, but systematic measurement might reveal how to slow it—extending information half-lives through careful experimental design, just as we learned to isolate and handle radioactive materials safely despite their persistent instability.