Summary
After their success with the AlphaGo program, having defeated grand master Go player Lee Sedol, DeepMind is moving on to the next big challenge, StarCraft II. Google’s Artificial Intelligence research group, DeepMind in partnership with Blizzard Entertainment, recently released a toolkit for artificial intelligence including an API to create automated scripts for the computer game StarCraft II.
There has been some previous research done on the original StarCraft. Every year student AI tournaments are held to show off the current, however even the best of these AIs wouldn’t stand a chance versus an average player and is not even comparable to a high-level human. So DeepMind and Blizzard are looking to further AI by training neural networks in order to learn how to compete with high-level players in the incredibly complex world of StarCraft II.
The research paper
The research paper released as an open-source collaboration between DeepMind and Blizzard introduces the API, provides a dataset of anonymous game replays and some initial analysis for neural networks trained on the dataset. The paper presents a fantastic toolkit to bring about more advanced AI research for StarCraft II.
Ultimately, the researchers intend for anyone to be able to utilize this API and dataset. Hopefully we will see more progress with the dataset of replays of human games and it might be useful in building systems capable of playing the game. For now these AIs are still in their infancy, but how long will it be until an automated system solves StarCraft II?
What is StarCraft II?
StarCraft II is a Real Time Strategy (RTS) game released by Blizzard Entertainment in 2010. The game revolves around three distinct races Zerg, Terran and Protoss each having unique units, buildings and strategies. The objective of the game is to eliminate your opponents main base. To accomplish this the player must accumulate resources (minerals and vespene gas), to construct production buildings in order to amass an army, while racing against your opponent who is making similar decisions.
A tournament match between two Grand master players from 2012.
What this means
Along with a research paper detailing a few implemented machine learning strategies, Blizzard and DeepMind released:
- An API for creating scripts and AI
- An image-based API (feature layers)
- Documentation, sample code, and sample bots
- Support for offline AI vs. AI play
- A replay pack of anonymous 1-on-1 matchmaking games
Together, DeepMind and Blizzard have made another step forward for research in artificial intelligence and machine learning. This is the first time that Blizzard has made one of their game APIs publicly available ever.
Replay of an untrained (left) and trained (right) neural net.
Why StarCraft II?
StarCraft II is a very complex and extremely competitive strategy game that runs in real-time. So each and every unit has multiple actions (IE attack, move, collect resource, build) and each building has multiple purposes, costs and bonuses. To add to the complexity there are 3 races (Zerg, Terran, Protoss) that have unique units, buildings and strategies.
The most common game mode is a 1-on-1 competitive showdown between two players. Online matchmaking filters players into seven different ranks starting at the lowest (Bronze) up to the highest (Grand Master), which is limited to 200 players in each region.
The lowest matchmaking tiers (not included Diamond, Master and Grand Master).
What makes StarCraft II so complex?
Another interesting feature, “Fog of war” introduces the concept of imperfect information where each player can only see the portion of the map where they have units or buildings. If you want to understand and react to your opponent it is necessary to use units to scout.
Additionally each game is largely dependent on economy and involves resource management with trade-offs for expansion and defense, and everything takes place on a large action space requiring hierarchical actions. However even basic features of the game present challenges to a completely blank AI, such as the strengths and weaknesses of three asymmetrical races, weighing up long-term goals against short-term micro wins and the real-time simultaneous actions of player and foe.
As a StarCraft II player myself, I averaged around a rank of Gold, reaching a peak rank of Platinum. And each game required so much focus and concentration that if you were to make a single mistake at any point in the game it could snowball.
So you can see StarCraft II is a very complex, challenging, messy, unsolved space similar to the real world, but as a game provides a simpler success criteria for learning. And so I believe it makes an excellent choice for machine learning research.
References
https://deepmind.com/research/alphago/
https://www.theatlantic.com/technology/archive/2016/03/the-invisible-opponent/475611/
https://www.blizzard.com/en-us/
https://github.com/deepmind/pysc2
https://arxiv.org/abs/1708.04782
Brayden's Blog 