John J Hopfield And Geoffrey E. Hinton received the Nobel Prize in Physics on October 8, 2024 for his or her research Machine learning and neural network algorithms that help computers learn. Her work was fundamental to the event of neural network theories underlying generative artificial intelligence.
A neural network is a computational model consisting of layers of interconnected neurons. As the Neurons in your brainThese neurons process and send information. Each neural layer receives a chunk of information, processes it and passes the result to the following layer. At the tip of the sequence, the network has processed the information and refined it into something more useful.
Although it could appear surprising that Hopfield and Hinton received the physics prize for his or her contributions to neural networks utilized in computer science, their work is deeply rooted within the principles of physics, particularly in a subfield called ” Statistical mechanics.
As computational Materials scientistI used to be pleased that this area of research was recognized with the prize. Hopfield and Hinton's work has allowed my colleagues and I to explore a process called generative learning for materials science, a way that underlies many popular technologies comparable to ChatGPT.
What is Statistical Mechanics?
Statistical mechanics is a branch of physics that uses statistical methods to clarify the behavior of systems composed of enormous numbers of particles.
Instead of specializing in individual particles, researchers use statistical mechanics to take a look at the collective behavior of many particles. Seeing how all of them work together helps researchers understand the system's large-scale macroscopic properties, comparable to temperature, pressure and magnetization.
For example, Physicist Ernst Ising developed a statistical-mechanical model for magnetism within the Twenties. Ising imagined magnetism because the collective behavior of Atomic spins interact with their neighbors.
In the Ising modelthere are higher and lower energy states for the system, and the fabric is more prone to exist in the bottom energy state.
A key idea in statistical mechanics is that Boltzmann distributionwhich quantifies how likely a certain condition is. This distribution describes the probability that a system is in a certain state based on its energy and temperature – be it solid, liquid or gaseous.
Ising accurately predicted the phase transition of a magnet using the Boltzmann distribution. He determined the temperature at which the fabric modified from magnetic to non-magnetic.
Phase changes occur at predictable temperatures. Ice melts into water at a certain temperature since the Boltzmann distribution predicts that the water molecules usually tend to assume a disordered – or liquid – state when heated.
Atoms arrange themselves in materials specific crystal structures that use the least energy. When cold, water molecules freeze into ice crystals with low energy states.
The same applies in biology: Proteins fold into low-energy shapeswhich permit them to act as specific antibodies – like a lock and key – against a virus.
Neural networks and statistical mechanics
Basically, all neural networks work based on the same principle – energy minimization. Neural networks use this principle to resolve computational problems.
For example, imagine a picture made up of pixels where you’ll be able to only see a part of the image. Some pixels are visible while the remaining are hidden. To determine which image it’s, consider all of the ways the hidden pixels might fit along with the visible parts. From there, you’ll select amongst all possible options from the almost definitely states based on statistical mechanics.
Veera Sundararaghavan
Hopfield and Hinton developed a theory of neural networks based on the thought of statistical mechanics. Just like Ising before them, who modeled the collective interaction of atomic spins to resolve the photography problem with a neural network, Hopfield and Hinton imagined collective interactions of pixels. They represented these pixels as neurons.
Just like in statistical physics, the energy of a picture refers to how likely a selected pixel configuration is. A Hopfield network would solve this problem by finding the bottom energy arrays of hidden pixels.
But unlike statistical mechanics – where energy is set by known atomic interactions – neural networks learn these energies from data.
Hinton popularized the event of a method called backpropagation. This technique helps the model determine the interaction energies between these neurons, and this algorithm underlies much of contemporary AI learning.
The Boltzmann machine
Building on Hopfield's work, Hinton imagined one other neural network called Boltzmann machine. It consists of visible neurons that we will observe and hidden neurons that help the network learn complex patterns.
In a Boltzmann machine you’ll be able to determine the probability that the image will look a certain way. To find this probability, you’ll be able to summarize all possible states that the hidden pixels might be in. This gives you the general probability that the visible pixels are in a selected arrangement.
My group worked on it Implementing Boltzmann machines in quantum computers for generative learning.
In generative learning, the network learns to generate latest data samples which are just like the information that researchers fed to the network for training. For example, it could generate latest images of handwritten numbers after being trained on similar images. The network can generate these by sampling from the learned probability distribution.
Generative learning is the premise of contemporary AI – it enables the generation of AI art, videos and text.
Hopfield and Hinton have significantly influenced AI research through the use of tools from statistical physics. Her work draws parallels between the best way nature determines the physical states of a fabric and the best way neural networks predict the probability of solutions to complex computer science problems.
image credit : theconversation.com