Side Track: Augmenting Text Adventures with Illustrative Art

Ryan Murdoch recently published another one of his cool projects in which he uses OpenAI’s CLIP to steer BigGAN towards matching textual inputs. The outputs have a surreal-expressionist quality to them which makes them interesting to look at. So the idea struck me to use them for text adventure illustration. I have taken Zork (which is now public domain), pushed its prompts into Ryan’s model and used this to augment the gaming experience. Here are a few examples:

You are facing the south side of a white house. There is no door here, and all the windows are boarded.

You are in the living room. There is a doorway to the east, a wooden door with strange gothic lettering to the west, which appears to be nailed shut, a trophy case, and a large oriental rug in the center of the room.Above the trophy case hangs an elvish sword of great antiquity.

The door reluctantly opens to reveal a rickety staircase descending into darkness.

As you can see, the images will not be detailed enough to turn the text adventure into a point-and-click adventure, but they certainly enhance the experience with interesting portraits.

It is also interesting to play with the number of input sentences. Let me demonstrate that as a final observation – depending on the number of sentences and iterations on the following prompt, you get images as diverse as the following three examples.

You are in the kitchen of the white house. A table seems to have been used recently for the preparation of food. A passage leads to the west and a dark staircase can be seen leading upward. A dark chimney leads down and to the east is a small window which is open. On the table is an elongated brown sack, smelling of hot peppers.