Digital approaches to the language of Shakespearean Tragedy

This post supplies data and further diagrams for a paper to be published in the forthcoming Oxford Handbook of Shakespearean Tragedy, edited by Michael Neill and David Schalkwyk.

You can download our main spreadsheet, with details of all plays included in the study, and frequencies for Docuscope LATs as either a csv file or an Excel file.

591 play corpus csv file

591 play corpus xls file

And here are distribution box plots for some of the features we discuss in the paper. Each box plot gives the distribution of one LAT in the entire corpus of printed drama (555 plays). Frequency values are along the horizontal axis, with number of plays corresponding to each frequency score plotted on the vertical axis. The shaded areas indicate where tragedies are placed within the overall corpus. The whisker plot above the bar chart shows outliers (black dots for tragedies, grey for other plays). Note how the distributions for these LATs in tragedies are shifted to the right, indicating an increase in frequency (and note the tendency for outliers to be tragedies).

LT boxplot TR Anger LT boxplot TR Fear LT boxplot TR Negativity LT boxplot TR Stds Neg LT boxplots TR Sad

 

TR v rest Autobio TR v rest First Person TR v rest Self Disclosure

Posted in Uncategorized | Leave a comment

Adjacencies, Virtuous and Vicious, and the Forking Paths of Library Research

Folger Secondary Stacks, western view

Folger Secondary Stacks, western view

Browsable stacks – shelves of books that you can actually look at, pull of the shelf, read a while, and put back. They’re wonderful. Folger readers regularly comment on the fact that they can walk freely through the stacks of the secondary collection, which in our case means books published after 1830. That collection is arranged by Library of Congress call number, and many know the system intuitively after years of library work. (I frequently find myself in the PRs and PNs.)

Recently I was looking through section PN6420.T5 for books on early modern proverbs, a topic I have been writing about for years. I was looking for Morris Palmer Tilley’s collection, A Dictionary of Proverbs in England in the Sixteenth and Seventeenth Centuries (Michigan: University of Michigan Press, 1950). There it was, right where it was supposed to be: a landmark piece of scholarship that is the first source for anyone interested in the topic. Yet this was only the first stop. On the shelves above and below this important source were about 30 other books on the subject, some of which I began to explore. Some very useful books turned up next to the one I had initially intended to find. Some of them have even turned up in my footnotes, the ultimate test, perhaps, of a book’s usefulness to a scholar.

Stack browsers are on the lookout for this kind of happy accident. You go into the stacks looking for this book, but another, more interesting, happens to be nearby. Now you can have a look, nibble around the edges of the promising title, which is an excellent form of procrastination if you are stuck or unready to begin writing. Having done my share of meandering in open stacks, I am intrigued when readers describe these moments of discovery ­– which after all are part of the natural progression of research – as happy accidents or the products of chance. Aren’t accidents things that you cannot, by definition, bring about or encourage?

The fact remains that libraries is set up to make such accidents happen. They arrange books on the shelves in a certain way – not at random, but on a plan designed to increase the likelihood that, nearby the book you think you want, there will be others you also want to read. When someone says, “and then I happened upon this great book,” they may be describing the advantages of the library’s structured arrangement of books by (say) subject matter. Partly an effect of a classification system, partly one of the physical arrangement of the space, Libraries are designed to promote “lucky finds.”

Such “encouragable accidents” are really the consequence of a simple principle that governs the entire space of the library: that of structured adjacency. As I will try to show in a moment, this principle can be seen at work in both the physical spaces of the stacks and the digital discovery spaces designed to give us access to the collection. The root of the word adjacency is the Latin verb jacere, which means to throw. When books appear side by side on a library shelf, their adjacency is not a product of chance: they have been placed (hopefully not thrown) together so that one is next to another of similar kind. How might one structure such adjacencies? One technique would be to shelve books by size. In some medieval monasteries, books of a similar size were placed on the same shelf. In addition to saving shelf space (think about it), this arrangement located collection access in the mind of the librarian or keeper who knew where different titles were. These collections weren’t designed to be browsed, so the principle made sense.

Now think of a modern, browsable stack of books arranged along the Library of Congress call number model. Here the principle of access exists in two places: the launching point of the card catalogue (which tells you where in the stacks to start looking) and then on the shelves themselves, where books on similar subjects are grouped together. The idea here is to use the intellectual scaffolding of subject cataloguing to structure the physical space of the collection. With respect to subject, physical adjacencies on the shelf become virtuous instead of vicious.

What is a virtuous adjacency? It is a collocation of two items likely to appeal to any-user-whatever whose item search is itself structured along principles which the cataloguing supports: usually author, date, title, subject, although there are many other forms of search. It doesn’t matter who you are or how deep your knowledge of the subject is: if you know enough to find one book on proverbs, you can find many in the Library of Congress system, because you are helped along by the arrangement in the physical space of the library. That arrangement is principled and intentional. It is virtuous.

But every virtuous adjacency can quickly become vicious, and this is because virtue (as I’m calling it) resides in the principles that inform any given reader’s search for a book. Suppose I know about Tilley’s book on proverbs, and I know it by title. Once I am pointed to that book by the catalogue, I go and look at it, and I see some terrific proverbs about apes, for example, “To make her husband her ape.”  I start to think about this. Maybe what I’m really interested in is how the behavior of apes helps people think about the nature of mimicry and mimesis in the early modern period. (Early modern references to apes are often veiled references to the mimetic power of artists, who “ape” nature.)

Proverb from Tilley's A Dictionary of Proverbs in England

Proverb from Tilley’s A Dictionary of Proverbs in England

Now the principle that governs the space flips. What I need to do is go to H. W. Janson’s magnificent Apes and Ape Lore in the Middle Ages and Renaissance, which has the call number GR730.A6 J3. What made the first adjacency surrounding “books about proverbs” virtuous was the collocation of books in space by subject. That was where the manufactured serendipity happened. But now that very principle of adjacency has become an impediment – it has become vicious – because Tilley is not surrounded by books about apes. I could search again under the latter subject, but that would not be adjacency, it would be search. We advert to catalogues in order to re-orient ourselves within the physical universe of books-on-shelves, or the virtual space of digital collections. But we cannot simply wander into that next thing that meets our new interest. To do this, I really would have to be lucky: “Oh look, there’s Jansen’s book on apes, just lying across the aisle….”

The moral of this story – or is it the proverb? – is that “every virtuous adjacency is also vicious.” When it comes to the arrangement of books, virtue is relative: it depends upon what the researcher thinks he or she is looking for, a thinking that often changes in the course of research. Once you’ve flipped from proverbs to apes, the physical arrangement of books on shelves is not going to help you. The virtuous arrangement that allowed you to lay your hands on that first book (“hey, my favorite book on proverbs!”) is now working against you (“shouldn’t I be looking at books about apes?”).

As we gain greater access to the contents of books; as digitized books and their machine actionable contents become more and more arrangeable with the assistance of mathematical principles like the topic model, the physical space of search is being transformed into something more plastic, even Borgesian. While the physical space of the library cannot be re-plotted whenever the research forks out onto another garden path, researchers have more options in the virtual space of text searching to find cut-throughs. There is a problem here, of course, which is that in such a virtual world of association, there are infinite pathways for association. It becomes more challenging to figure out where to go next when you could go anywhere.

But there may be other ways to multiply virtuous pairings given the tools that librarians of the future will create. Instead of starting with Tilly and then hoping that I’ll be lucky enough to bump into Jansen, I might rely on my mobile device to reach into the contents of the book I’m interested in now and, based on a principle of adjacency I supply, rearrange all the books in the library around that first book in concentric layers of immediacy of different types – layers that might allow readers to move from one virtuous adjacency to the next. There is no way around the virtuous/vicious symmetry, since it is precisely that symmetry which makes research necessary: in exploring the connection between these five books on proverbs, you are giving up the opportunity to think about that other, really, really good book about apes. (You can tell I wish I’d found Jansen earlier.) What makes an adjacency for one research task virtuous makes that adjacency vicious for the next.

That’s why answers to research questions do not turn up instantly. You have to decide when to shift directions, and the physical layout of library stacks according to a single principle of adjacency (e.g., subject cataloguing) is going to sustain some inquiries while simultaneously shutting down others. No amount of dynamic text search is going to put an end to the virtuous/vicious circle: their pairing represents a real constraint on knowledge – the fact that thinking is progressive, and moves on discrete pathways – rather than a technological or physical limitation to be overcome.

That is not to say that there aren’t new ways of mapping adjacencies among digitized texts. Abstract models of the contents of books such as topic models, however, do offer us other pathways in the research process; they are an additional principle of adjacency that we can invoke if we don’t want to “jump the hedge” by consulting a book’s footnotes (say) and then searching for new items based on the titles referenced there. (On topic models, see Ted Underwood’s very helpful blog post.) We have been using topic models in the Wisconsin VEP project to look at our collections of texts, and they do seem to open up adjacencies that we would never have thought about. (An upcoming blog post will deal with the relationship between the novel and English moral philosophy.) A topic model can suggest, for any given book or passage, another book or passage that might be relevant for reasons only a user could recognize (but might not be able spontaneously to supply). As with other techniques of dimension reduction (e.g., PCA, factor analysis), there may be more topics than we can name or recognize: a topic does not become a principle of association until a human being recognizes and affirms that principle in action.

If libraries are gardens with many forking paths, the hedges that separate those paths are absolutely real. Even a fully virtual, instantly re-arrangeable virtual rendering of our shelf spaces will not put an end to vicious adjacencies, since they too will become virtuous if research takes a new turn.  Our challenge is not a physical one; it’s not even computational. In a future library where any two books could be placed alongside one another in an instant, we might never find anything we want to read.

The task of library research is not simply that of poking around clusters of items on a shelf, or more grandly, finding ways of reclustering books continuously in hopes of finding the ultimate, virtuous arrangement. There is no Leibnizian, maximally virtuous arrangement of books, and never will be. (Leibniz must have hit upon this melancholy thought when he was librarian at Wolfenbuttel.)

But there are more or less definite lines of thought, each on its way to becoming other, equally definite, lines of thought. There is no point in celebrating the fact that such lines can fork off in an infinite number of directions. We know already that a researcher can only follow one of them at a time.

Posted in Uncategorized | Tagged , , , , | Leave a comment

Quantification and the language of later Shakespeare

img-5

 

The written version of a paper we gave in Paris last year (2013) has just been published by the Société française Shakespeare. Here is the paper (which is in English), and here are the citation details:

Pour citer cet article

Référence papier

Jonathan Hope et Michael Witmore, « Quantification and the language of later Shakespeare », Actes des congrès de la Société française Shakespeare, 31 | 2014, 123-149.

Référence électronique

Jonathan Hope et Michael Witmore, « Quantification and the language of later Shakespeare », Actes des congrès de la Société française Shakespeare [En ligne], 31 | 2014, mis en ligne le 29 avril 2014, consulté le 07 mai 2014. URL : http://shakespeare.revues.org/2830

Posted in Early Modern Drama, Shakespeare, Uncategorized, Visualizing English Print (VEP) | Tagged , , , | Leave a comment

Hamlet in five words

g2g Hamlet

 

Farah Karim-Cooper asked us to write something for the Globe to Globe Hamlet site. Here it is.

Posted in Early Modern Drama, Shakespeare | Tagged , | Leave a comment

Scotland’s Collections and the Digital Humanities

On 2nd May 2014 I’m presenting at the second event in this series, entitled ‘Working with Data’. This post is intended mainly for those who come to the session as a record of links I’ll mention, and a resource for those starting out in text analysis. It may also be useful for others as a collection of material.

UPDATE: This is Mia Ridge’s page of resources for ‘Data Visualisation for Analysis in Scholarly Research‘, a course she teaches at the British Library (and updates regularly). The list is excellent. Twitter: @mia_out

from my presentation:

http://blogs.ucl.ac.uk/transcribe-bentham/             - crowdsourced, TEI transcriptions

http://www.textcreationpartnership.org/tcp-eebo/             - University funded double keyed transcriptions, TEI

http://www.tei-c.org/index.xml                            - Text Encoding Initiative: standards, training, resources, other projects

http://earlyprint.wustl.edu                                      - examples of visualisations

http://voyant-tools.org                                             – some basic text analysis tools

http://www.wordle.net                                            - self-confessed ‘toy’ for word-clouds

http://mallet.cs.umass.edu                                   – more serious set of text analysis tools

http://ucrel.lancs.ac.uk                                         – Lancaster University’s UCREL site: wide range of corpora and tools including

http://ucrel.lancs.ac.uk/vard/about/                - automatic modernisation of spelling

 

 

Resources for text analysis

0          Introductions/anthologies available on the web

0.1       Literary Studies in the Digital Age: An Evolving Anthology, edds Kenneth M. Price, University of Nebraska, LincolnRay Siemens, University of Victoria

 

http://dlsanthology.commons.mla.org

 

Alan Liu, “From Reading to Social Computing”

David L. Hoover, “Textual Analysis”

Susan Schreibman, “Digital Scholarly Editing”

Charles Cooney, Glenn Roe, and Mark Olsen, “The Notion of the Textbase: Design and Use of Textbases in the Humanities”

Stéfan Sinclair, Stan Ruecker, and Milena Radzikowska, “Information Visualization for Humanities Scholars”

      William A. Kretzschmar, Jr., “GIS for Language and Literary Study”

                   Tanya Clement, “Text Analysis, Data Mining, and Visualizations in Literary Scholarship”

Julia Flanders, “The Literary, the Humanistic, the Digital: Toward a Research Agenda for Digital Literary Studies”

Daniel Powell, with Constance Crompton and Ray Siemens, “Glossary of Terms, Tools, and Methods”

 

0.2       A Companion to Digital Literary Studies (Blackwell)

http://www.digitalhumanities.org/companionDLS/

 

0.3       A Companion to Digital Humanities (Blackwell)

http://www.digitalhumanities.org/companion/

 

0.4       Misc: Matt Jockers http://www.matthewjockers.net/2013/01/03/advice-for-dh-newbies/

 

Ryan Cordell, ‘doing digital humanities’  http://ryan.cordells.us/s13dh/

 

http://prezi.com/r7rmqbxifpq9/introducing-digital-humanities-full/?utm_source=twitter&utm_medium=landing_share

 

 

1          Text Analysis

 

1.1       TextAnalytics 101 by John Laudun (@johnlaudun)

 

http://johnlaudun.org/20130221-text-analytics-101/

 

A very clear guide to basic text analytics, and why numbers might get you and your students into the language of texts.

 

The piece will set you up to do basic analysis with Python scripts if you want, but you don’t need to do this to follow the argument, which deals thoughtfully with the ‘why bother?’ of text analytics. Highly recommended.

 

1.2       Where to start with text mining

 

http://tedunderwood.com/2012/08/14/where-to-start-with-text-mining/

 

Slightly different tack from Laudun, as stresses the need to compare large numbers of texts. Very clear about basic principles.

 

1.2       Lisa Spiro: introduction to text analysis, powerpoint presentation – a good overview with further links, should be understandable even without the talk

 

http://digitalscholarship.wordpress.com/2013/01/31/slides-and-exercises-from-doing-things-with-text-workshop/

 

1.3       Michael Ullyot: data curation and an overview of text analysis – another excellent overview, specifically on Early Modern material, but relevant generally

 

http://ullyot.ucalgaryblogs.ca/2012/09/07/data-curation-in-the-networked-humanities/

 

1.4       Text Analytics: higher level debates on The Waves

 

http://cforster.com/2013/02/reading-the-waves-with-stephen-ramsay/

from this graduate DH class http://630dh.cforster.com/syllabus/

 

 

2          Network analysis

 

2.1       Map your facebook network with Gephi: a tutorial

 

http://pegasusdata.com/2013/01/10/facebook-friends-network-mapping-a-gephi-tutorial/

 

 

2.2       scottbot on hartlib correspondence: heatmap and network visualisations

 

http://scottbot.net/uploads/map/demo/maps_heatmap_layer/hartlibEurope.html

 

 

 

 

3          Literary History

 

3.1       DH is changing what literary history is – and suggesting that we don’t actually know what it is. Here is Ted Underwood (@Ted_underwood) on the rise and fall of first person in the novel:

 

http://tedunderwood.com/2013/02/08/we-dont-already-know-the-broad-outlines-of-literary-history/

 

and see other posts at http://tedunderwood.com

 

3.2       For a discussion of ‘influence’, and links to Matt Jockers’ work, see

 

http://winedarksea.org/?p=1629

 

3.3       Proportions of male/female pronouns:

 

http://sappingattention.blogspot.ru/2013/02/genders-and-genres-tracking-pronouns.html

http://sappingattention.blogspot.co.uk/2013/02/canonic-authors-and-pronouns-that-they.html

 

 

5          Critique

 

5.1 Wendy Hui Kyong Chun

 

http://www.c21uwm.com/2013/01/09/the-dark-side-of-the-digital-humanities-part-1/#more-348

http://chronicle.com/blogs/conversation/2013/01/05/on-the-dark-side-of-the-digital-humanities/

http://www.queergeektheory.org/2013/01/mla13-the-dark-side-of-digital-humanities/

http://www.dispositio.net/archives/1340

 

 

6          Lists/surveys of tools:

 

6.1       Jeffrey McClurken (University of Mary Washington) http://mcclurken.org/   : guide to digital history.

 

https://docs.google.com/document/d/1GIV0xQOl5ZNHdguTZy7mpECsil6mu_ZqkZXjhjpE0eg/edit?pli=1

 

 

6.2       http://dirt.projectbamboo.org/

“Bamboo DiRT is a tool, service, and collection registry of digital research tools for scholarly use. Developed by Project Bamboo, Bamboo DiRT makes it easy for digital humanists and others conducting digital research to find and compare resources ranging from content management systems to music, OCR, statistical analysis packages to mindmapping software.”

 

6.3       http://journalofdigitalhumanities.org/about/

“The Journal of Digital Humanities (ISSN 2165-6673) is a comprehensive, peer-reviewed, open access journal that features the best scholarship, tools, and conversations produced by the digital humanities community in the previous quarter.

The Journal of Digital Humanities offers expanded coverage of the digital humanities in three ways. First, by publishing scholarly work beyond the traditional research article. Second, by selecting content from open and public discussions in the field. Third, by encouraging continued discussion through peer-to-peer review.”

 

6.4       http://digitalhumanitiesnow.org/

“Digital Humanities Now showcases the scholarship and news of interest to the digital humanities community through a process of aggregation, discovery, curation, and review. Digital Humanities Now also is an experiment in ways to identify, evaluate, and distribute scholarship on the open web through a weekly publication and the quarterly Journal of Digital Humanities.

WHAT WE PUBLISH

Digital Humanities Now highlights work from the open web that has gotten the attention of the digital humanities community or is worthy of such attention, based on critical editorial review. Scholarship—in whatever form—that drives the field of digital humanities field forward is highlighted in the Editors’ Choice column. Additional news items of interest to the field—jobs, calls for papers, conference and funding announcements, reports, and recently-released resources—also are redistributed.”

 

6.5       http://selection.datavisualization.ch/

“Datavisualization.ch Selected Tools is a collection of tools that we, the people behind Datavisualization.ch, work with on a daily basis and recommend warmly. This is not a list of everything out there, but instead a thoughtfully curated selection of our favourite tools that will make your life easier creating meaningful and beautiful data visualizations.”

6.6       40 Essential Tools and Resources to Visualize Data

http://flowingdata.com/2008/10/20/40-essential-tools-and-resources-to-visualize-data/

 

 

 

 

 

Jonathan Hope/May 2014

Posted in Uncategorized | Leave a comment