Information Theory: Introduction to Coding

Through practice, Alice and Bob discover their wire system supports a maximum of two plucks per second—exceeding this rate causes confusion and detection errors.

Bob proposes replacing continuous voice signals with discrete wire plucks—high-energy impulses easily distinguished from background noise.

Alice and Bob’s communication reduces to transmitting dice roll outcomes—messages selected from a finite set of 11 possible symbols (numbers 2 through 12).

Alice and Bob initially adopt the simplest encoding strategy: transmitting dice results as the literal count of plucks matching the rolled number.

The video asserts that Alice’s probability-based coding strategy achieves provable optimality—no alternative encoding using identical plucks can reduce average transmission time further.

Alice designs an optimal encoding strategy: assign shortest signals to most probable symbols, allocating longer signals to rare outcomes.

Alice recognizes that two-dice roll outcomes follow a predictable probability pattern: one way to roll 2, two ways for 3, three ways for 4, continuing to six ways for 7 (most common), then symmetrically decreasing.

Alice and Bob face the fundamental communication problem: distinguishing intentional signals from environmental noise interference in their wire-based communication system.