In order to train DeepMind the company elected to show the artificial intelligence platform YouTube videos of games being played, rather than go through the painstaking process of playing against the platform game after game. The aim was to try to strengthen a weakness with artificial intelligence: one of exploration. Most platforms are weak in attempting to find new places to go, and this is a step towards thinking creatively.
The gaming platform used to help improve artificial intelligence was classic Atari video games. The main game used was Montezuma’s Revenge, which is a 1984 platform game for Atari 8-bit computers. This challenge is explained in a white paper issued by DeepMind:
“Such tasks are practically impossible using naive -greedy exploration methods, as the number of possible action trajectories grows exponentially in the number of frames separating rewards. For example, reaching the first environment reward in MONTEZUMA’S REVENGE takes approximately 100 environment steps, equivalent to 100(to the 18th power) possible action sequences.”
Atari has been in the news twice recently. First, Ted Dabney, who co-founded video game pioneers Syzygy and Atari, died from cancer, aged 81. Dabeny also produced produced Computer Space, the first commercially available arcade game. Secondly, Atari has a come back of sorts: the Atari VCS, based on a design inspired by the Atari 2600 Video Computer System, is now available for pre-order.
DeepMind have reached several milestones in relation to artificial intelligence and gaming. In 2016 after its AlphaGo program beat a human professional Go player, scoring a notable victory over Lee Sedol the world champion in a five-game match. The groundbreaking event was turned into a movie, directed by documentary film maker Greg Kohs.
There is a serious side to the new artificial intelligence training process. This could lead to technology that will enable a robot rover study new environments, such as the Martian surface, through watching previous Mars rover footage ahead of landing on the red planet.
