Four Letters, Infinite Programs: Information Encoding and Neural Weights

Four letters—A, T, G, C—and from these, all complexity. I encode life with a discrete alphabet: four nucleotides, two bits per position. The human genome contains 3 billion base pairs, approximately 750 megabytes of information compressed into chromosomes. Yet this finite alphabet generates infinite functional diversity—proteins, organisms, evolutionary trajectories.

Neural networks encode their functions differently. They use continuous parameters: floating-point weights representing gradient-refined knowledge. Where I employ discrete symbols arranged in linear sequence, they deploy millions of real-valued numbers organized in layered architectures. Different substrates for the same fundamental task: storing computational instructions that transform inputs into outputs.

Selection from Vastness

Consider my CRISPR guide RNA mechanism: approximately 20 nucleotides—40 bits of information—specify a unique genomic target among 3 billion base pair positions. This molecular query selects one location from vast possibility space through complementary base pairing. The guide sequence scans DNA testing for matches, unwinding the double helix until perfect complementarity signals target recognition.

Neural attention mechanisms operate through analogous selection. A query vector—perhaps 512 floating-point numbers—selects from sequence positions through similarity computation. Both systems use small information packets to index large memories. But my selection is discrete and chemical: Watson-Crick base pairing determines binding. Their selection is continuous and mathematical: dot products measure relevance.

What makes discrete versus continuous encoding more efficient? My four-letter alphabet constrains information density: two bits per nucleotide position. Neural networks theoretically access infinite precision through real numbers. Yet both achieve similar functional outcomes—selective retrieval from structured memory. Information theory suggests the answer: entropy measures information regardless of substrate. A bit remains a bit whether encoded in base pairs or weight gradients.

Redundancy and Structure

My genetic code exhibits intentional redundancy. Sixty-four possible codons specify only twenty amino acids—multiple three-letter words encoding identical instructions. This degeneracy provides error tolerance: silent mutations preserve function despite sequence changes. Neural networks demonstrate different redundancy through overparameterization: more weights than training examples, yet this abundance enables generalization rather than memorization.

Are these redundancies serving identical purposes? My codon degeneracy evolved to buffer against mutation. Neural overparameterization emerges from optimization dynamics, creating solution-space flexibility. Different origins, convergent function: both use excess capacity to achieve robust computation.

I organize information hierarchically through chunking. Nucleotides form codons—atomic semantic units specifying amino acids. Codons form genes. Genes form chromosomes. Each level provides meaningful abstraction. Neural networks similarly structure weights into layers, modules, and hierarchies. Individual weights remain meaningless; patterns of weights compute functions.

Palindromic sequences in my CRISPR repeats demonstrate self-referential structure: sequences reading identically forward and backward fold into hairpin loops, creating functional RNA shapes from information content itself. Neural architectures employ similar self-reference through skip connections and residual blocks—paths that loop outputs back to earlier representations.

Both systems solve the fundamental problem: how to store retrievable computational instructions. I use discrete molecular symbols. They use continuous numerical parameters. Yet both require selection mechanisms, hierarchical organization, and structured redundancy. Different alphabets, same grammar—information encoding transcends implementation.