The World is Gonna Know Your Name

This week's post involves William Rowen Hamilton, so I thought I'd piggyback on another Hamilton's popularity! This week I saw a news article about improving neural networks, like the one I talked about a few weeks ago, by adding information from the system's Hamiltonian, which characterizes the total energy available.

Like the Newtonian and Lagrangian techniques, the Hamiltonian is yet another completely equivalent way to determine system dynamics. In most cases, the Hamiltonian is defined as the sum of kinetic and potential energy

where the quantities are defined in terms of some coordinates q, and corresponding momenta p. Then the equations of motion are given by

As with the Lagrangian, this can often be a simpler way to define a system where energy is conserved.

The paper that the group wrote concerns chaotic systems, where small changes to conditions can cause wildly different results. Often a chaotic system will have a subset of simple results – I actually wrote about an example long ago, but since I hadn't yet taken any nonlinear dynamics, I wasn't sure if it qualified. If you're trying to train a neural network to predict a system's dynamics, this transition between simple and chaotic motion can cause problems.

One of the example systems the paper studies is a circular billiards table, with a peg in the center:

Figure 8

A simple solution could be a ball bouncing between the peg and the wall on a radial line. If we gave the ball a random direction though, chances are it would bounce around the circle in a complex pattern. Those dynamics would be entirely deterministic, but impossible to perfectly predict in a real-world situation.

The group's idea was to change the way the neural network is trained, incorporating the Hamiltonian to give it awareness of the chaotic nature of the system. They compared the results from a standard network, a Hamiltonian network, and from solving the differential equations:

Figure 9

From top to bottom, the cases are non-chaotic hitting the outer circle only, non-chaotic hitting both boundaries, and chaotic. While the Hamiltonian network is far from an exact match to the true trajectory, it does a much better job characterizing the behavior of the ball with the same amount of training. By giving the network a rule to conserve energy, we can teach it a little bit of physics to better train it (potentially to become our robot overlords)!