Banditopod

Recently as part of my research, I've been trying to measure a probability distribution – specifically, the chances that we've seen a certain signal in LISA. The trouble is, there are many random noise factors that go into the calculation of whether we see the signal or not, so it's not a straight equation I can plug things into. Instead we need to sample it many times to estimate the distribution, and this can be expensive. My colleague Henri suggested I could use a technique called Markov Chain Monte Carlo (MCMC). I thought to get a better feel for the method, I'd try out a simple example here.

There's a traditional problem in probability theory called the "two-armed bandit." Imagine a slot machine with two levers – You insert a coin and choose which lever to pull. Each arm has a certain probability of paying out, but the only way to find out is by playing, and looking at how often you win or lose. What then is the best strategy for choosing a lever? You may have gotten lucky your first few pulls of one lever and overestimated its chances of winning.

We can make this more like my research by extending to a multi-armed bandit – Each arm represents a set of parameters we're searching for, and we want to pick the arm with the biggest payout/best fit to the data. Still to be answered though is how we pick which arm to play: Imagine a set of players, who can choose an arm at each step based on the wins/loses they've seen. Each one is more likely to pick an arm with lots of wins, but might try another arm just in case. Now, if we look at the estimated probabilities for each arm as time goes on, we might think we'd get a good idea of the true values:

The blue line is the true probability for each arm, and the orange dots are the estimates based on the average number of wins. The dots are jumping around so much though that it's hard to see how well we're doing. Instead of animating in time, we can try looking at how frequently we play each arm:

Pretty quickly, each arm gets a consistent rate of pulls, but it looks like we're undersampling the highest-probability arms. I think this may be due to the top-probability arms having fairly similar values – As I pointed out above, we can't tell whether we have the best lever, or just a streak of luck, so we hedge our bets. A common technique with MCMC is run a "burn in" for a while to let the players move around the parameter space, then reset the probability estimates and continue running.

As a final view of the data, we look at how the players distribute themselves among the arms through time [NB: The x-values are off by 1 compared to the earlier plot due to the way I gave the distribution to the MCMC tool I used]:

It starts off fairly flat – the parameter exploration I was talking about – but after a certain point, the distribution establishes itself, and from there the shape simply scales upward. However, even if you could afford to play tens of thousands of times, I think you'll be hard-pressed to find a slot machine paying out as frequently as these!

Tactical Sailing

Marika is an avid sailor, and former Fleet Captain of the University of Michigan Sailing Club. In hearing about her exploits on the high seas (or Huron River, as the case may be), I've often thought about the mechanics of sailing. Specifically, I was curious if I could model the forces involved in maneuvers like tacking and jibing.

The model I came up with was to consider two connected forces on the boat: wind hitting the sail, and drag from the water. When wind hits the sail, the air bounces off, imparting momentum in the direction of the surface of the sail. The magnitude of the momentum transferred will be proportional to the dot product of the sail direction and the wind direction – maximum is when the wind hits the full face and zero when it's perpendicular. The drag from the water will be similar to the wind hitting the sail, but this time we're looking at the shape of the hull for each bit of water that hits the boat. Both these forces depend on and change the velocity of the boat, so we iterate the calculation until we find the equilibrium. To do that, we need to come up with a model for the hull of the boat.

I couldn't find any existing mathematical models for hull shape, but looking at common designs, I came up with this concept: Take two halves of an ellipse with semi-axes a and b, and put them a distance W apart. As long as W is less than 2a, the halves will intersect at each end. Now measure L from one end, and cut the remainder off. Using numbers for the width and length from here, and picking a and b that seemed reasonable, here's what I came up with:

Now to the simulation: We consider a wind going from left to right and try different orientations of the boat and sail. For each case, we start with zero velocity and calculate forces from wind and drag. Those forces give a new velocity, and we repeat until there's little change. The final velocity is shown by the arrow. Now, I didn't want to spend a lot of time figuring out the exact momentum transfer for the wind and drag, so I fudged the scaling to get reasonable results. Even with this super simple model though, we can see cases where it's possible to sail into the wind! (Look for the arrow pointing left.)

As with all of my posts, I don't recommend trying this yourself based on my calculations, but I'm hoping Marika will give me some experimental experience in the future!

In My Corner

For a while I've been interested in analyzing the game Jenga, since it's very physics-aligned in its design: Maintaining balance under changing forces. I couldn't get a handle on how to look at it though, until I connected it to a tool that's used frequently in my research field: corner plots. Corner plots are a way to display correlations between different variables in a large data set. For gravitational waves, they're used to show how estimates of a source's parameters depend on each other, such as a black hole's mass and spin. I figured I could come up with some measurements of Jenga games, and look at how they relate to each other.

First, we need a way to collect some data. As a reminder, here's what a Jenga tower looks like:

Wikipedia

Each level has 3 blocks that alternate in direction. On a turn, we remove a block from anywhere below the top level, and then add it to the top. The tower will fall and end the game if the center of mass of a subset is over an empty space, and the side is not supported. I was able to make a simulation of this in Python, with the virtual player making a random move on each turn, so long as that move does not cause the tower to fall. Eventually, the player will run out of moves and the game will end, but we can look at what kinds of moves result in longer games. I struggled a bit to find a way to show an example tower being built, and settled on a side view with all the bricks facing out. Remember that the bricks actually alternate in direction, so a hole under a brick is not necessarily a problem:

Now to get our statistics, we want to run a large series of games, and get some measurements from each one. The parameters I chose were max height of the tower, number of turns taken, fraction of turns for which the center block of a level was removed, fraction of turns a block was placed on the center of the top, and the fraction of times a block was removed from the upper half of the tower. After running 500 simulated games, we can make a corner plot with the results (click to enlarge):

This shows us some interesting connections between our parameters. There is strong correlation between the maximum height and the number of turns – This makes sense, since each turn we put a brick on the top level. We can also see anti-correlation (negative slope) between removing the center brick and max turns/height. This is because once the center of a level is taken, we can't take either of the sides, so we run out of bricks faster. Our final two variables, adding to the center and removing from the upper half, don't seem to have much effect on the outcome, indicated by the circular distributions.

Now in reality, Jenga bricks are designed with some irregularities to make the balance a little more difficult to predict, so this is another case where knowing the physics might not help you win, but it was interesting way to explore a tool for visualizing data.