[Side Track] Video Games with AI Angle Part 1: Turing Test, Scanner Sombre and Gabriel Knight 3

Turing Test

While Portal 2 is more focused on navigating 3D spaces and Q.U.B.E. is more focused on dexterity, this game puts strong emphasis on actual reasoning in relatively confined, well-discretized spaces with very few dexterity challenges. For instance, here is a video of the last puzzle.

In the last puzzle you have a switch that toggles a door lock (i.e. one door is always open, one closed). You also have a magnet and an energy beam powering it – when you interrupt the beam, the magnet turns off. Furthermore, there is a conveyor belt going in a circle passing this energy beam and a metal box. So you notice that if you throw the box onto the conveyor belt, it will interrupt the energy beam at regular intervals. Now the problem is that the switch for the goal door you want to exit through ultimately is behind the door lock and can only be operated by your robot companion you can switch into – all you need to do is flip this switch and exit through the door.

However, in order to do that you have to combine all these ideas and have one a-ha moment: If you place the magnet over the switch and place a metal object (i.e. box or robot) on it and the other one on the conveyor belt, the magnet will pull up the object on the switch most of the time, but drop it and thus trigger the door lock whenever the energy beam is interrupted by the other object on the conveyor belt. The problem is: If your robot is on the conveyor belt or pulled by the magnet, it cannot simultaneously pass the door lock. So the a-ha moment is that you yourself have to use your body as a tool. You’re not magnetic, so you cannot be the object on the switch. Hence, you throw the metal box on the switch, jump onto the conveyor belt, switch into the robot, pass the door lock when your human body passes the energy beam (and thus triggers the magnet-door-button-cascade) and can flip the switch to then transform back into yourself and exit. I just loved that puzzle in the context of what we’re doing – the reasoning process I went through doing this would be tremendously cool to reproduce in a software system.

Another aspect I like about the game is that in many corridors there are optional side puzzles. When you solve them, you usually gain entrance to an extra room with additional reading about Turing, Searle and (iirc) Dennett. It is also an interesting setting: The puzzles exist to protect humans from AIs – according to the setting, only human creativity can solve this collection of puzzles. Here is a full walkthrough in case someone wants to see everything:

Scanner Sombre

Essentially, this game is on the list because of the Johansson point light figure and its inspiration for cognitive science. The game is primarily about navigating game a game world comprised of point clouds, but the ending is interesting in the point light figure context.

Gabriel Knight 3

Point & click adventures are not only one of my favorite game genres, but due to their strong puzzle background they are also interesting in terms of AI. While most adventures require the combination of items in the player’s inventory as well as using them with scene objects, there are lots of intriguing puzzle variants and Gabriel Knight 3 utilizes a lot of them. The rough setting is that you are in a hotel with a whole cast of mysterious characters in the French city of Rennes-le-Château. While most of them are on a treasure hunt, it is evident that there has to be much more to it. The game stands in the long templar conspiracy tradition from Umberto Eco’s excellent Foucault’s Pendulum over the rather grotesque book Holy Blood, Holy Grail over another very famous adventure game series called Broken Sword (which is worth a post in itself) to superficial mainstream literature like Dan Brown. But instead of going into the (historically most likely not too interesting story) around Abbe Sauniere whose sudden wealth sparked conspiracy theories around the French city the game is set in, let me first provide the video and then get into the puzzles.

As you can see from glancing over the video, you analyze maps, gather geophysical subsurface imaging data, investigate crime scenes, sneak up on characters to eavesdrop in on conversations, listen with drinking glasses on doors, disguise yourself, illegally enter hotel rooms to investigate them, explore a largely open world, analyze paintings, decipher a cryptic poem (Le Serpent Rouge), play with multiple characters, interpret text, conversations and the actions of other agents in the environment and you learn about the rich history of the setting which includes the extensive use of a wiki to explore new topics among many other elements. While this game is still clearly out of reach for contemporary AI agents, it does provide an interesting example of which kinds of challenges I would ultimately like to solve. While playing arcade games – e.g. with DQNs like the early Atari 2600 work by Mnih et al. – is intriguing [later DeepMind’s Agent57 even reached superhuman performance on all Atari 2600 games and Gato, A Generalist Agent which Scott Reed et al. released just yesterday (fun fact: I sat at Scott Reed’s prior desk at UMich for almost 2 years) looks marvelous as well], it does not bridge the gap to higher levels of reasoning, whereas point and click adventures require a fluidly intelligent solution that dynamically combines various forms of skills and spectrums of reasoning which to me feels like, pun intended, a holy grail of AI. In the AI Painter experiment I gave a brief example of what a very simple first step on tackling adventures might look like against the much simpler Maniac Mansion, but Gabriel Knight 3 might be the ultimate challenge for adventure playing AIs. If I ever find some free time, I would love to play with it – after all, that’s what games are for.

Part 2 can be found here.

0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply