Elon Musk’s ‘Dota 2’ AI bots are taking on pro teams

OpenAI took on individual players at last year’s The Invitational in a one-on-one minigame, and pros said that by watching the matches back, they were able to learn from the bots. But playing as a team introduces different types of intricacies, and OpenAI had to teach the AI how to coordinate the five bots.

At any time, a hero (or character) can make one of around 1,000 actions; the bots have to make effective decisions while processing around 20,000 values representing what’s going on in the game at a given time. The average number of possible actions in chess is 35, so this is a little more complex than the Deep Blue supercomputer that beat chess grandmaster Garry Kasparov in the ’90s.

To teach its bots what to do, OpenAI uses reinforcement learning. That’s essentially a trial-and-error method, where, over time, the AI evolves from completely random behavior, to a more focused style of play. OpenAI runs Dota 2 on more than 100,000 CPUs, and the AI plays itself to the tune of 180 years’ worth of games every day. In just a few hours, the bots can play more games than a human can in a lifetime, giving the AI ample opportunity to learn. But machines learn in different ways from humans, so it’s not an apples-to-apples comparison. Otherwise, the AI team would have been the best in the world in a snap.

Dealing with the game’s bi-weekly updates is a challenge, as those can shift gameplay mechanics. Since the field of view is limited to what’s on screen, the AI needs to make inferences about what the other team is doing and make decisions based what it thinks its opponents are up to. The bots have some advantages, such as an 80ms reaction time, which is faster than humans are capable of. They also perform around 150-170 actions per minute, which is comparable to top human players.

There are some limits on the AI though. The bots only use five of the game’s 115 heroes and play against a team made up of the same characters. Some decisions are made for them by humans, like which skills to level up in. OpenAI developers also restricted some items and cut off some of the game’s more intricate aspects like invisibility and warding, which lets players snoop on other parts of the map.

OpenAI started playing against amateur teams recently, and so far the bots have beaten or drawn with every opponent, including one in the top one percent of the semi-pro ranks. As a warmup for the exhibition match at The Invitational, OpenAI is playing a top team July 28th, and is streaming the battle on Twitch. Eventually, OpenAI hopes have bots and humans playing together on the same team, as they’re capable of different things.

The goal for OpenAI not necessarily to create an unbeatable Dota 2 squad that takes home the lion’s share of The Invitational’s $15 million prize pool, but instead to learn how the training method can help AI solve all kinds of complex problems. In the meantime, OpenAI will heave to be content with beating just about any Dota 2 team — as long as there are some restrictions in place.