Hopfield Nets and Statistical Mechanics
摘要
Having familiarized ourselves with the basic theory and practice of Hopfield nets, we can now widen our perspective. In this chapter, we therefore look at Hopfield nets from the point of view of physics or, more specifically, statistical mechanics. The basic theme will be to understand a Hopfield net as an ensemble of interacting microscopic entities and to apply statistical techniques to reason about their collective behavior or macroscopic state. First, we discuss Ising models of spin systems and relations to Hopfield nets to answer the still open question of why we may talk about the energy of a Hopfield net. Based on these foundations, we then look at stochastic Hopfield nets or Boltzmann machines and the idea of stochastic asynchronous updates. Finally, we will get to know completely different ways of running Hopfield nets and study stochastic simulated annealing and mean field annealing.