For part 2, I’d like to cover a tutorial in matplotlib that is our boid GUI. As far as my Googling skills could take me, I couldn’t find an animation similar to what I was looking for: two separate scatter plots that logically trace each other in an endless loop. The closest I could find Read more about Matplotlib Animation (Boids Part 2)[…]
The next few posts will be about Boids. From the Wikipedia page: Boids is an artificial life program, developed by Craig Reynolds in 1986, which simulates the flocking behaviour of birds. His paper on this topic was published in 1987 in the proceedings of the ACM SIGGRAPH conference.  The name “boid” corresponds to a Read more about High Performance Computing (Boids Part 1)[…]
The capstone project of my Real Analysis class was to pick a topic and learn it independently. I chose contraction mapping because of its importance in dynamic programming and optimization problems. I am grateful to many keen bloggers who wrote out more thorough proofs than many textbooks. Below is my contribution to the internet: a Read more about Banach’s Fixed Point Theorem[…]
The main problem of empirical analysis is comparing the right “apples to apples”. The Sharpe Ratio is a statistical method for converting oranges to apples. Check out my example code on Github here. The higher the Sharpe Ratio, the greater the returns per unit of risk. Notes 1.) I briefly mention in the Jupyter Notebook Read more about Sharpe Ratio[…]
I was fortunate to attend OpenAI Five today and play on stage in San Francisco, streamed live on Twitch. I was floored by the performance of OpenAI bots. The following is my dig on the bots and Dota.
The bots seemed to approach the game differently than the humans. While we (the bots and the humans) both know that the goal of the game is to destroy the ancient, we approach that goal differently. For example,
except: group as five
As opposed to
except: last hitting creeps
Is the above a true simplification of the process? I checked OpenAI’s website and found this. This page details the values given to the bots’ reward functions. It appears the numbers are randomly generated at first, but over large empirical datasets, the optimal value is found. One of the important things to notice is the priority given to buildings. In contrast, last hitting is small. However, let’s take a look at the total weight given to buildings versus last hitting when accounting for gold and xp earned.
This is the basic formula for 1 pre 5 min creep wave with no denies.
This is the basic formula for 1 T1 tower.
From the bot match I played, we lost our first T1 around 3 mins. The bots were efficient, only trading about 1 wave for 1 tower. They move quickly from the objective back to lanes or to the next objective. On napkin style math, it appears a perfectly CS-ed wave is worth a little more than a T1 tower. One element I did not weigh, however, is what OpenAI calls “team spirit”. Bots value their teammates scores very highly. How often have I played a pub where my safelane carry refuses to TP because “it wouldn’t be worth it for me to miss this wave”? In contrast, bots are happy to rotate early (their mid laner left the lane at level 4 to secure T1 top). It is not in their code to push as 5, but it is learned that cooperation supersedes individual success. This was also seen by the cores letting supports last hit in lane.
So what about game 3 in casters vs bots? Why did the bots lose so badly? One of the problems Purge noted is the bots were not prioritizing farming other lanes. They would cut creeps and hit towers, but they weren’t TP-ing away as soon as they saw an enemy. Slark was close to his shadow blade for so long, yet didn’t clear some camps to buy it. My brother noted this as a fatal flaw of generality. The bots were not able to adapt to poor game situations and therefore are not as good as he thought. I am here to defend our future overlords (may the bots remember my piety).
By not letting the bots draft their own heroes, it renders their empirical dataset useless. The reward functions are specifically created to maximize their probability of winning a game given certain assumptions– such as drafting. Purge asks why they don’t finish their items and split the map better. It’s because they can’t. It is not in their learned behavior of how to win games. In a situation where they have their optimal lineup, it would be a waste to farm a lane and finish an item. One of my friends argues that changing the draft conditions is as severe as adding a new hero. It changes the immensely complex landscape that is Dota 2 and the bots have never seen it before. The early OpenAI blog posts tell us the first few agents wandered around the map aimlessly before even learning to last hit. Similarly, the bots in game 3 stayed in trees or die backed because they didn’t know what to do.
In summary, given a model the bots have trained, they will always win. Change the fundamental assumptions of that model, and sure, humans will win.
A piece of advice I used to give followed the lines of “If I managed to do it… so can you!”. In retrospect, that was terrible advice because I perpetuated survivorship bias. A younger student at my college recently confessed to me their admiration for my successes and their hopes to manifest as something akin Read more about HackerWrecked[…]
The other day I was thinking about the long term costs of poverty. An example is dental care. Dental care is generally not covered for adults on Medicaid, though it is covered for children. However, dental health is just as important as other forms of health. Not only can poor dental health be painful and Read more about Interest Rate on Poverty[…]
Today’s post is a “flex” on trying out Tableau and working again with ProPublica’s Data Store. Below is a graphic on federal government spending at Trump Properties since 2015, created from this dataset. The legend is displayed as Amount Property Number of records **ProPublica puts a disclaimer that government agencies have fought to not disclose Read more about Federal Government Spending at Trump Properties[…]
I only recently discovered ProPublica and their free datasets. One that caught my eye was a file that compiled the top selling products on Amazon by their price and seller. The article that was produced using this dataset, “Amazon Says It Puts Customers First. But Its Pricing Algorithm Doesn’t” says Amazon uses default features for affiliated Read more about Amazon Pricing Algorithm[…]
This page is currently under construction. Check back soon for more updates!