Generative Model That Won 2024 Nobel Prize

Ludwig Boltzmann’s statistical physics framework and modern probabilistic models share a core idea: the probability of a system’s state depends exponentially on its energy and the temperature of the environment.

Boltzmann machines generalize deterministic Hopfield networks by updating neurons probabilistically according to local energy differences tied to the Boltzmann distribution.

Boltzmann machines and related energy-based models learn via a contrastive Hebbian rule that balances data-driven association with suppression of unrealistic “dreamed-up” states.

Boltzmann machines gain expressive power from hidden units that capture abstract features, and restricted Boltzmann machines (RBMs) introduce structural constraints that make training tractable.