Multiplayer ALE

[benblack769]

Atari has some great multiplayer games. This PR starts the process of supporting these games with:

• a new clean API for handling multiplayer games that is 100% backwards compatible
• Support for 4 new games: Combat, Joust, Maze Craze, Warlords
• support for 2 player modes for 20 games, including 16 old games and the 4 new ones
• support for 4 player modes for Warlords and Pong

New API

The new api has the following methods in addition to all the old ones (in both the python and c++ interfaces):

• numPlayersActive() -> int
  • the number of players in the current mode
• act(list of actions) -> list of rewards
  • overloads act(int) ->int, which is still there
  • Requires that the number of actions matches numPlayersActive()
• allLives() -> list of lives
• getAvailableModes(num_players=1) -> list of modes
  • returns modes that support the specified number of players.
  • List is empty when the number of players is not supported at all

In addition, there are a couple of other changes to ALE functionality

• Default mode is not 0 always, instead it is set automatically to be the first mode in the lowest number of players supported (1 player for every environment except Combat). I.e. Surround sets mode to be 2 automatically because mode 2 is the first 1 player mode.
• Some changes to mode numbering systems in some of the games
• Initial actions are also performed by player b (needed for double dunk)
• The lives() of many of the environments is changed to fit the interpretation: 0 lives means the game is over if you die. 1 life means you have 1 more chance if you die, etc.

Merge process

This merge process will likely be lengthy, so we expect to host a fork of ALE ourselves in the medium term, but we hope we can get various stakeholders excited about the potential of multiplayer ALE.

[JesseFarebro]

Could you maintain a separate branch apart from your main project that's rebased on the current head? This is a little too much to take a look at in its current state.

[jkterry1]

@JesseFarebro We'll do this, but it's going to take a little while, and we need to fix something in entombed cooperative first.