Log in
/images/sticker-top.png

BLOG

Scoffing at Mere Generation

You may have read a story called The Library of Babel by Argentine writer Jorge Luis Borges. The library of the story is a fantastically vast construction of hexagonal rooms, containing every book that was ever written, as well as every book that can, or will, ever be written. The library is finite but huge: it contains every book that can be written with a 25 character alphabet in 410 pages at 80 characters per line and 40 lines per page. It’s easy to imagine such a library, and easy in principle to construct one too, but the library is not as useful as it might seem. For every good book hiding on the shelves, there are countless millions of nonsense books containing random text, and worse, countless millions of half-nonsense books that mix real insight with random nonsense. There is no definitive catalogue for the library, but since a catalogue is itself just a book, the library contains millions of books claiming to be catalogues, almost all of which are gibberish, while many are just misleading. 

Borges showed us that extremes are easy: to generate everything we just need an alphabet and a simple way to combine them, and to generate nothing we need just do nothing. The real challenge is to generate something in between, something with meaning and literary merit that is not random and not a rehash of what someone else has already written.

 Simple ways to combine letters or methods that cut up existing texts and put them back together in random orders are called mere generation, generation for the sake of it. Though the results might occasionally be seen as creative by an outside person, the generator itself can’t tell the good outputs from the nonsense, just like Borges’ library has no way of separating the good books from the bad. What if we start with an existing book by a respected author, and make scripted changes to this book to obtain a new one that is different but meaningful? We call this pastiche if the new text remains close enough to the original to successfully piggyback on its meaning.  When we build software systems to be creative like humans, it is tempting to build ones that specialize in mere generation or in pastiche, because these two extremes are the easiest to implement. They also need the least amount of knowledge about the world to be built in to the software.

We scoff at mere generation in Computational Creativity because it is precisely the kind of generation that does not require creativity. Or rather, it does not require creativity on the part of the generative mechanism itself. It shifts this need for creativity onto the audience, who must now sift through the outputs of the system like a prospector panning for gold, finding just a few grains of gold dust in every large pile of mud and silt. 

 

A good example of mere generation on the Web can be found on Slate magazine here. This little embedded app generates new internet "rumours" about the upcoming Star Wars episode VII by simply generating permutations of the actor names, character names and plot details that have already been leaked. Here is a sample output:

Unnamed sources tell us that Star Wars Episode VII will focus on a Padawan learner (played by Adam Driver) who must learn the ways of the Force at the hands of an aging Luke Skywalker. Domhnall Gleeson and Max von Sydow will play the children of Han Solo and Princess Leia, and our intel tells us that together they will go up against a fearsome Sith Lord (played by Oscar Isaac), along with a menacing motion-capture alien (played by Andy Serkis). Lastly—though we were sworn to secrecy—we can now say with a high degree of confidence that the sequel will be called Star Wars Episode VII: Moff Tarkin Strikes Back.

Note the framing here, the textual structure into which the combinatorial elements are randomly plugged. A Mere Generator often relies heavily on this kind of framing to carry the combination and to imply more meaning than is actually present.

The idea of mere generation is an old one. As evidenced by the story of Borges, it certainly predates the field of Computational Creativity. It is an idea commonly expressed via the Infinite Monkey Theorem. As this old chestnut puts it, an infinite number of monkeys (or other animals capable of battering the keys of a typewriter) working for an infinite amount of time will eventually produce all the great works of literature produced by human creators over the ages. We can think of Borges' library as having been created by an army of such monkeys.

This seems a pretty clear-cut notion when presented in such an extreme form: near-infinite libraries of noise, or infinite armies of monkeys producing gibberish. But mere generation is a much more subtle notion in Computational Creativity. How do we say with certainty that one approach, or one system, is merely generative while another transcends mere generation to to be thoughtful, insightful and "creative".

Well, if a machine produces output whose quality it cannot estimate or ascertain for itself, and requires a user to do this filtering for it, then that system is merely generative. But what if the system uses some rudimentary internal ranking and filtering, perhaps based on a statistical model of what humans have liked in the past? Suppose this system produces less noise and more signal, but still requires its human users to hold their noses and carefully filter the good from the bad, or the usable from the unusable. Is this system still merely generative?

It seems therefore that mere generation is not a Boolean either-or notion. It is a continuum, from dump-truck generators at one end that obey rules to generate outputs that must always be filtered by humans, to more self-confident and self-critiquing systems at the other end that constrain their outputs and exhibit some sense of their own quality.

How can we tell systems apart from different points on this continuum? A system of the first kind might confidently (and falsely) proclaim itself to be of the latter kind. Or a system of the latter kind may hide its complexity so well that users assume it is performing no self-critiquing at all. Do we need to show users the inner workings of the system? Is that even possible?

One solution is to have a system explain its choices to the user, to say why it selected one output and rejected another. Only by talking about its own outputs can we see whether a system was creative in generating them, or whether it was slavishly following enigmatic rules.

Of course, this explanation generation is itself a creative task that can be faked using mere generation. Where does it all end?

Your comments on this issue -- which is perhaps the defining issue of our field -- are very welcome!

ps: the Infinite Monkey Theorem was lampooned on a Simpson's episode here.

Mr. Burns: This is a thousand monkeys working at a thousand typewriters. Soon, they'll have finished the greatest novel known to man.

[reads a page]

Mr. Burns: All right, let's see... "It was the best of times, it was the BLURST of times?" You stupid monkey.

Comments

In (partial) defence of Mere Generation

Comment: 

It should be noted that mere generation is frowned upon in CC for good reason, but that does not make mere generation an intrinsically bad idea. In other areas of computation, such as internet art, digital humanities, etc., where the goal is to showcase the insights and ingenuities of the human programmer, mere generation can be a very useful (and entertaining) device. See for instance the work (and published works) of CC researcher Nick Montfort of MIT.

babel.py

Comment: 

Here is a Python script that can produce any conceivable page (A4) of text. Inconceivable! I call it babel.py. It will naively attempt to recognise pages that appear to make sense in English (i.e., including frequent words such as "the" and "I") but you have to be patient... very patient. It's like an inverse Turing-test: do you have more patience than the script? Your time might be better spent writing a better script to defeat this baseline! from random import choice, randint frequent = set(( "the", "be", "to", "of", "and", "a", "in", "that", "have", "I", "it", "for", "not", "on", "with", "he", "as", "you", "do", "at" )) # In English, 30% of words are nouns, 14% verbs and 11% punctuation. # This leaves 45%, including 10% determiners such as "the" and "a". # Generate A4 pages with random words, # looking for those that have about 45% of words in the frequent list: THRESHOLD = 0.45 def word(n=6): return "".join(choice("abcdefghiklmnopqrstuvwxyz") for i in range(n)) while True: s = "" for i in range(randint(300, 400)): # A4 n = randint(1, 10) # English avg word length = 5-6 s += word(n) + " " s = s.split() n = len(s) * 1.0 i = sum(1 for w in s if w in frequent) # Be patient! if i / n >= THRESHOLD: print " ".join(s)

Add new comment

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.