Arts

Introduction

Like literary studies, the field of art history also explores how to provide digital access to art works and develop new modes of analysis.

Although literary studies have benefitted from linguistics in terms of models and tools for text analysis, the visual arts, including sculpture and video, present greater challenges to the development of analytical models. Although there certainly are plenty of categories that may be formalized (abstract/figurative, light/dark, etc.), there are at least as many challenges to the development of an analytical approach to art as there are to analyzing literature.

The presence of art in literature assumes two principal forms: there is the use of imagery in books and digital media, often restricted to cover art, but sometimes also used in the text itself, as is the case with W. G. Sebald’s novel, Austerlitz, where numerous photographs are an integral part of the work. Computational visual analytics could improve the understanding of what books are often judged by—their covers—are they becoming more complex? Brighter? More figurative?

Secondly, art is also present as a subject, and by describing a work of art in words, also known as ekphrasis. Many of the chapters in Georges Perec’s Life—A User’s Manual (1978), include ekphrasis, and a key theme of the book are aquarelles that are made into puzzles. It is also impossible to think of Marcel Proust’s In Search of Lost Time without considering its detailed reflections on art. In both cases, because of the works’ complexity, it makes sense to use computational assistance, just as one would if the roles of art and ekphrasis were studied at a vast scale.

Applications

Elementary

One way to computationally approach art is to investigate how it is represented as a topic, for instance by counting references to visual art in literature. Doing so in an automated way can be complicated due to the huge variations in the representation of art in literature - there are many different ways in which you can refer to art and art serves different purposes in different contexts. Observing keywords in context might therefore be an interesting angle to approach arts computationally. For instance, Voyant tools offer a Key Word In Context (KWIC) concordance function allowing you to investigate co-occurrences of words or characters throughout one or several literary works. Additionally, counts derived manually from a larger corpora of several books can be compared computationally. 

The digitization of arts allows you to analyse simple metrics such as colour. Doing so could be another way to approach art computationally, and there are several freely available online tools allowing you to visualize or interact interactively with literature and art of different kinds that might be useful for this purpose. Some of these resources are made for specific literary works, but can nonetheless provide inspiration to how these approaches could be applied to other literary works as well. You can for instance go to Yale DHLab and get inspiration from the project Quantizing Color in William Blake’s Illuminated Books, which is a project that has generated a visualization tool called BlakeTint that is useful for investigating changes in Blake’s color palette across books. BlakeTint will be launching for public use shortly. 

Another project at DHLab at Yale University investigated Vogue Covers in the project Vogue Covers in Colormetric Space. They used the freely available visualization software ImagePlot to quantitatively visualize the colorfulness of Vogue covers from the 1890s to the 2010s.

Advanced

To deepen your analysis, automated methods can be used to quantify and confirm initial observations in the visualized data. Deep learning approaches have been used to detect genre-specific characteristics in book covers (see e.g. Buczkowski et al., 2017). Apply the dataset and scripts used for the article “Judging a Book by its Cover” (Iwana et al., 2016)  or make your own analysis using the Open Library Cover API. Build a convolutional neural network (CNN) algorithm with Python and Keras and train your model (for underlying computational theory, see this tutorial). Do you find genre-specific features of visual aesthetics?

Another cross section of literature and art are comics where pictures are in the narrative core. Have a look at the Computational Approaches of Comic Analysis (Laubrock & Dunst, 2020), use one of the datasets mentioned in the article, and carry out your own experiment on stylistic classification of graphic novels or character and object recognition in them, using similar deep learning methods mentioned above.

Of course, there are a lot of other ways art and literature are related (see e.g. projects of Yale DHLab), so be creative and think of novel ways of interdisciplinary digital explorations.

Resources

Scripts and sites

  • Digital Art History, an online textbook and collection of resources for digital analysis of art history.

Articles



Performance »