Digital Literary Studies – A Companion Guide

 AU About Aarhus University  Topics Creative Writing

Creative Writing

creative fiction screenshot
GPT-3 Creative Fiction: William Shakespeare’s Sonnet 18, reimagined and rewritten by an artificial intelligence. Branwen, G. (2020, June 19), Gwern.net.

Introduction

Creative writing has a long history of being imagined as left to computers. In George Orwell's Nineteen Eighty-Four, songs and novels for the masses are written by kaleidoscopic machines—this does not make much sense as a technique, but the vision of automated writing is clearly present.

Today, we can be certain that a lot of information in newspapers and on the Internet has been created by machines that deliver stock quotes, sports news, and so on. But that is not creative writing, and although translation has come a long way towards automation, the prospect of a computer writing first-rate literature seems far off. As impressive as the GPT-3 algorithm is in its ability to create something that stays on topic and could pass for human writing, it still has its flaws and lacks the inherent human creativity. However, these techniques are still under development, and there will certainly be many other attempts to create writing machines that have been trained on large text archives. While algorithms won’t replace human writers in a long time, they might become useful assistants for the writing process.

More importantly, electronic literature exists as a genre that has never quite taken off with a broad audience, but which explores new ways of creating and reading text, sometimes dependent on the input from the reader, and sometimes creating reading experiences that break radically from "normal" reading on paper or a screen.

Finally, there are authors who use computers in their writing process. The Danish poet Klaus Høeck, whose works are freely available online in John Irons' translations, experimented early with making programs - in Algol - that would challenge him to be creative in a particular way. As such, this is an extension of selecting other kinds of restrictions - the form of a sonnet, a list of things to be included in a novel as Georges Perec did - and taking on the computer's complexity and opacity as an element in the creative process.

Applications

Elementary

Delve into the world of electronic literature to explore the vast array of opportunities that technology and digital tools provide the field of creative writing. You can find inspiration at Electronic Literature Organization, dedicated to the investigation of writing and publishing as well as reading of literature in electronic media. If you browse their Electronic Literature Collection Volume 3, you will come across publications such as the project Collocations, an interactive work of digital poetry built upon two key texts from Albert Einstein’s and Niels Bohr’s historical debates about quantum physics. Collocations is designed for tablets and responds to the reader’s position, movements, and orientation. Employing strategies of erasure, visual poetry, and algorithmically defined systems, Collocations lets the reader’s interactions with the device determine which words of the text become highlighted, consequently forming unique poetic texts (Avnisan, 2015). 

The above mentioned project illustrates how computational approaches to creative writing can be concerned with both writing and reading processes, although being fully or partly created by an algorithm is not necessarily a sufficient demarcation of creative writing. Nonetheless, computational approaches undoubtedly allow for new ways to create and read texts, and quite impressive creative writing machines have been made publicly available. One example is AI Dungeon, an online text adventure tool with a simple interface that can be accessed from your browser. Based on commands, comments, or directions given by the user, AI Dungeon interactively generates fiction using the impressive GPT-2 algorithm, one of the world’s most advanced AIs. A paid version of AI Dungeon built upon the strikingly more powerful GPT-3 algorithm was launched in 2020, opening up for unprecedented computational approaches to creative writing (Lamerichs, 2020).

Advanced

The basic approach to creative writing with computational methods is similar to recent methods in machine translation. For natural language processing (NLP) tasks, modern neural network models are first trained on a large corpus from which they learn linguistic features and patterns. Then the general pre-trained model can be fine-tuned for a specific task, using a dataset tailored for the purpose. In the case of creative writing, a pre-trained language model could thus be fine-tuned on novels of a certain genre to generate a new story in that writing style. 

One of the best performing models right now is GPT-3 by OpenAI (Brown, et al., 2020). Have a look at many projects created with it on GPT-3 Creative Fiction. The earlier version, GPT-2, is available for public use. If you want to train your own automated text generator, there are many tutorials, such as this one training GPT-2 in Python. Notice that training such a big model takes time and computing power. 

Another angle into creativity is to measure how creative a text is. Although algorithms are more apt at producing metrics than creating human-like, creative content themselves, the challenge with measuring creativity is the subjectivity and variability in the definition for creativity. One proxy for creativity is the semantic similarity of words: concepts that are usually not related in meaning are seen as a characteristic of novelty and creativity in a text. Following this line of thought, e.g. metaphors that use surprising word combinations, increase the creativity of a text. Other metrics used to capture creativity computationally are novelty, surprise, rarity, and recreational effort, based on the assumption that these notions can make a text stand out from the mass (Karampiperis et al., 2014). 

Try out measuring novelty in terms of word similarity with a word similarity scorer. Select two texts that use a particular word frequently, extract sentences where that word is used and upload the data with those sentences as responses to see how creative the combinations in the texts are. In your mind, do the results reflect creativity?

Resources

Scripts and sites

  • Avnisan, A. (2015). Collocations. An interactive and experimental creative writing project at the intersection of art, writing, science, and code.
  • Computational Creativity in Digital Humanities - A Primer.

Articles

Critical Writing » 
 

Revised 15.11.2022