The Time Problem: Rigid Classifiers, Classifier Postmarks

By Michael Witmore | Published: April 16, 2012

 

Here is a thought experiment. Make the following assumptions about a historically diverse collection of texts:

1) I have classified them according to genre myself, and trust these classifications.

2) I have classified the items according to time of composition, and I trust these classifications.

So, my items are both historically and generically diverse, and I want to understand this diversity in a new way.

The metadata I have now allows me to partition the set. The partition, by decade, items, and genre class (A, B, C)  looks like this:

Decade 1, 100 items: A, 25; B, 50; C, 25

Decade 2, 100 items: A, 30; B, 40; C, 30

Decade 3, 100 items: A, 30; B, 30; C, 40

Decade 4, 100 items: A, 40; B, 40; C, 20

Each decade is labeled (D1, D2 D3) and each contains 100 items. These items are classed by Genre (A, B, C) and the proportions of items belonging to each genre changes from one decade to the next. What could we do with this collection partitioned in this way, particularly with respect to changes in time?

I am interested in genre A, so I focus on that: how does A’ness change over time? Or how does what “counts as A” change over time? I derive a classifier (K) for A in the first Decade and use this distance metric to arrange all items in this decade with respect to A’ness. So my new description allows me to supply the following information about every item: Item 1 participates in A to this degree, and A’ness means “not being B or C in D1.” Let’s call this classifier D1Ka. I can now derive the set of all classifiers with respect to these metadata: D1Ka, D1Kb, D1Kc, D2Ka, D2Kb, etc. And let’s say I derive a classifier for A using the whole dataset. So we add DKa, DKb, DKc. What are these things I have produced and how can they be used to answer interesting questions?

I live in D1, and am confident I know what belongs to A having seen lots of examples. But I get access to a time travel machine and someone sends me a text written much later in time. It is a visitor from D4, and by my own lights, it looks like another example of A. So, I have projected D1Ka onto an item from D4 and made a judgment. Now we lift the curtain and find that for a person living in D4, the item is not an A but a B. Is my classifier wrong? Is this type of projection illegitimate? I don’t think so. We have learned that classifiers themselves have postmarks, and these postmarks are specific to the population in which they are derived. D1Ka is an *artifact* of the initial partitioning of my data: if there were different proportions of A, B, and C within D1, or different items in each of these categories, the classifier would change.

Experiment two. I live in D4 and I go to a used bookstore, where I find a beautifully preserved copy of an item produced in D1. The title page of the this book says, “The Merchant of Venice, a Comedy.” Nonsense, I say. There’s nothing funny about this repellent little play. So D1Ka fails to classify an A for someone in D4. Why? Because the classifier D4Ka is rigidly determined by the variety of the later population, and this variety is different from that found in D1. When classifiers are themselves rigidly aligned with their population of origin, they generalize in funny ways.

Wait, you say. I have another classifier, namely Ka produced over the entire population, which represents all of the time variation in the dataset of 400 items. Perhaps this is useful for describing how A’ness changes over time? Could I compare D1Ka, D2Ka, D3Kz and D4Ka to one another using DKa as my reference? Perhaps, but you have raised a new question: who, if anyone, ever occupies this long interval of time? What kind of abstraction or artifact is DKa, considering that most people really think 10 years ahead or behind when they classify a book? If we are dealing with 27 decades (as we do in the case of our latest big experiment), we have effectively created a classifier for a time interval that no one could ever occupy. Perhaps there is a very well-read person who has read something from each decade and so has an approximation of this longer perspective: that is the advantage of the durability of print, the capacity of memory, and perhaps the viability of reprinting, which in effect imports some of the variation from an earlier decade into a newer one. When we are working with DKa, everything is effectively written at the same time. Can we use this strange assumption — everything is written at once — to explore the real situation, which is that everything is written at a different time?

Another interesting feature of the analysis. This same type of “all written at the same time” reasoning is occurring in our single decade blocks, since when we create the metadata that allows us to treat a subpopulation of texts and belonging to *a* decade, we once again say they were written simultaneously. We use obvious untruths to get at underlying truths, like an astronomer using the inertial assumption to calculate forces, even though we’ve never seen a body travel in a straight line forever.

If classifiers are artifacts of an arbitrarily scalable partitioning of the population, and if these partitions can be compared, what is the ideal form of “classifier time travel” to use when thinking about how actual writing is influenced by other writing, and how a writer’s memory of texts produced in the past can be projected forward into new spaces? Is there anything to be learned about genre A by comparing the classifiers that can be produced to describe it over time? If so, whose perspective are we approximating, and what does that implied perspective say about our underlying model of authorship and literary history?

If classifiers have postmarks, when are they useful in generalizing over — or beyond — a lifetime’s worth of reading?

 

Posted in Counting Other Things | Tagged , , | 2 Comments

Google Books: Ratio of Inked Space to Blank Space

By Michael Witmore | Published: April 14, 2012

 

How could we create a proxy measure for the relative luxury of a book, and by extension the social prestige of its contents? One way of getting at this might be to measure the ratio of inked to non-inked space for a given work. While the measure is flawed  – verse uses less page space, illustrations may sometimes apply more ink across the page — it is at least a starting point. What if Google Books were to publish the ratio of inked to non-inked space for all of the items it has scanned? We could then see how writing of different types, for example, plays or prose fiction, move into larger print formats such as the Folio.

Posted in Counting Other Things | Tagged , , | 2 Comments

Shakespeare’s mythic vocabulary – and his invisible grammar

By Jonathan Hope | Published: February 14, 2012

Universities in the UK are under pressure to demonstrate the ‘impact’ of their research. In many ways, this is fair enough: public taxes account for the vast majority of UK University income, so it is reasonable for the public to expect academics to attempt to communicate with them about their work.

University press offices have become more pro-active in seeking out stories to present to the media as a way of raising the profile of institutions. Recently, the Strathclyde press office contacted me after they read one of my papers on Strathclyde’s internal research database: they wanted to do a press release to see if any outlets would follow-up on the story.

The paper they’d read was a survey article I’d written for an Open University course reader. My article reported recent papers by Hugh Craig and Ward Elliott & Robert Valenza, which demolish some common myths about Shakespeare’s vocabulary (its size and originality – and see Holger Syme on this too) – and went on to argue that Shakespeare’s originality might lie in his grammar, rather than in the words he does not make up.

Indeed they did want to pick up on the story, though I’d have preferred the article to have been a bit clearer, and not to have had a headline that was linguistic nonsense. The Huffington Post did a bit better.

One particularly galling aspect of the stories: the articles failed to attribute the work on Shakespeare’s vocabulary to Craig or Elliott and Valenza, so it might have looked as though I was taking credit for other people’s work

Looking back, I don’t think I explained my ideas very well either to Strathclyde’s press office, or to the Daily Telegraph when they rang – hence the rather confused reports. But I was extremely careful to attribute the work to those who had done it – even to the point of sending my original text to the journalist I talked to, and pointing him to the relevant footnote. I did not expect a news story to contain full academic references of course – but a clearly written story could easily have mentioned the originators of the work.

A minor episode, but it also made me think that there is a fundamental problem with trying to explain complex linguistic issues in the daily press – even if you use Newcastle United’s greatest goalscorers to illustrate the statistics. They want a clear story: you want to get the nuances across. Luckily, this blog allows me to make the full text of my article available (click through twice for a pdf of my article):

Shakespeare and the English Language

 

Jonathan Hope, Strathclyde University, Glasgow, February 2012

Posted in Counting Other Things, Early Modern Drama, Shakespeare | Tagged , , , , , , , , , , , | 1 Comment

The very strange language of A Midsummer Night’s Dream

By Jonathan Hope | Published: February 6, 2012

I just got back from a fun and very educative trip to Shakespeare’s Globe in London, hosted by Dr Farah Karim-Cooper, who is director of research there.

The Globe stages an annual production aimed at schools (45,000 free tickets have been distributed over the past five years), and this year’s play is A Midsummer Night’s Dream. I was invited down to discuss the language of the play with the cast and crew as they begin rehearsals.

This was a fascinating opportunity for me to test our visualisation tools and analysis on a non-academic audience – and the discussions I had with the actors opened my eyes to applications of the tools we haven’t considered before. They also came up with a series of sharp observations about the language of the play in response to the linguistic analysis.

I began with a tool developed by Martin Mueller’s team at Northwestern University: Wordhoard, as a way of getting a quick overview of the lexical patterns in the play, and introducing people to thinking statistically about language.

Here’s the wordcloud Wordhoard generates for a loglikelihood analysis of MSND compared with the whole Shakespeare corpus:

 


Loglikelihood takes the frequencies of words in one text (in this case MSND) and compares them with the frequencies of words in a comparison, or reference, sample (in this case, the whole Shakespeare corpus). It identifies the words that are used significantly more or less frequently in the analysis text than would be expected given the frequencies found in the comparison sample. In the wordcloud, the size of a word indicates how strongly its frequency departs from the expected. Words in black appear more frequently than we would expect, and words in grey appear less frequently.

As is generally the case with loglikelihood tests, the words showing the most powerful effects here are nouns associated with significant plot elements: ‘fairy’, ‘wall’, ‘moon’, ‘lion’ etc. If you’ve read the play, it is not hard to explain why these words are used in MSND more than in the rest of Shakespeare – and you really don’t need a computer, or complex statistics, to tell you that. To paraphrase Basil Fawlty, so far, so bleeding obvious.

Where loglikelihood results normally get more interesting – or puzzling – is in results for function words (pronouns, auxiliary verbs, prepositions, conjunctions) and in those words that are significantly less frequent than you’d expect.

Here we can see some surprising results: why does Shakespeare use ‘through’ far more frequently in this play than elsewhere? Why are the masculine pronouns ‘he’ and ‘his’ used less frequently? (And is this linked to the low use of ‘lord’?) Why is ‘it’ rare in the play? And ‘they’ and ‘who’ and ‘of’?

At this stage we started to look at our results from Docuscope for the play, visualised using Anupam Basu’s LATtice.

 

 

The heatmap shows all of the folio plays compared to each other: the darker a square is, the more similar the plays are linguistically. The diagonal of black squares running from bottom left to top right marks the points in the map where plays are ‘compared’ to themselves: the black indicates identity. Plays are arranged up the left hand side of the square in ascending chronological order from Comedy of Errors at the bottom to Henry VIII at the top – the sequence then repeats across the top from left to right – so the black square at the bottom left is Comedy of Errors compared to itself, while the black square at the top right is Henry VIII.

One of the first things we noticed when Anupam produced this heatmap was the two plays which stand out as being unlike almost all of the others, producing four distinct light lines which divide the square of the map almost into nine equal smaller squares:

 

These two anomalous plays are Merry Wives of Windsor (here outlined in blue) and A Midsummer Night’s Dream (yellow). It is not so surprising to find Wives standing out, given the frequent critical observation that this play is generically and linguistically unusual for Shakespeare: but A Midsummer Night’s Dream is a result we certainly would not have predicted.

This visualisation of difference certainly caught the actors’ attention, and they immediately focussed in on the very white square about 2/3 of the way along the MSND line (here picked out in yellow):

 

So which play is MSND even less like than all of the others? A tragedy? A history? Again, the answer is not one we’d have guessed: Measure for Measure.

This is a good example of how a visualisation can alert you to a surprising finding. We would never have intuited that MSND was anomalous linguistically without this heatmap. It is also a good example of how visualisations should send you back to the data: we now need to investigate the language of MSND to explain what it is that Shakespeare does, or does not do, in this play that makes it stand out so clearly. The visualisation is striking – and it allowed the cast members to identify an interesting problem very quickly – but the visualisation doesn’t give us an explanation for the result. For that we need to dig a bit deeper.

One of the most useful features of LATtice is the bottom right window, which identifies the LATs that account for the most distance between two texts:

 

This is a very quick way of finding out what is going on – and here the results point us to two LATs which are much more frequent in MSND than Measure for Measure: SenseObject and SenseProperty. SenseObject picks up concrete nouns, while SenseProperty codes for adjectives describing their properties. A quick trip to the LATice box plot screen (on the left of these windows):

 

confirms that MSND (red dots) is right at the top end of the Shakespeare canon for these LATs (another surprise, since we’ve got used to thinking of these LATs as characteristic of History), while Measure for Measure (blue dots) has the lowest rates in Shakespeare for these LATs.

So Docuscope findings suggest that MSND is a play concerned with concrete objects and their descriptions – another counter-intuitive finding given the associations most of us have with the supposed ethereal, fairy, dream-like atmosphere of the play. Cast members were fascinated by this and its possible implications for how they should use props – and someone also pointed out that many of the names in the play are concrete nouns (Quince, Bottom, Flute, Snout, Peaseblossom, Cobweb, Mote and so on) – what is the effect on the audience of this constant linguistic wash of ‘things’?

Here is a screenshot from Docuscope with SenseObject and SenseProperty tokens underlined in yellow. Reading these tokens in context, you realise that many of these concrete objects and qualities, in this section at least, are fictional in the world of the play. A wall is evoked – but it is one in a play, represented by a man. Despite the frequency of SenseObject in this play, we should be wary of assuming that this implies the straightforward evocation of a concrete reality (try clicking if you need to enlarge):

 

Also raised in MSND are LATs to do with locating and describing space: Motions and SpaceRelations (as suggested by our loglikelihood finding for ‘through’?). So accompanying a focus on things, is a focus on describing location, and movement – perhaps, someone suggested, because the characters are often so unsure of their location? (In the following screenshot, Motions and SpatialRelation tokens are underlined in yellow.)

 

 

Moving on, we also looked at those LATs that are relatively absent from MSND – and here the findings were very interesting indeed. We have seen that MSND does not pattern like a comedy – and the main reason for this is that it lacks the highly interactive language we expect in Shakespearean comedy: DirectAddress and Question are lowered. So too are PersonPronoun (which picks up third person pronouns, and matches our loglikelihood finding for ‘he’ and ‘his’), and FirstPerson – indeed, all types of pronoun are less frequent in the play than is normal for Shakespeare. At this point one of the actors suggested that the lack of pronouns might be because full names are used constantly – she’d noticed in rehearsal how often she was using characters’ names – and we wondered if this was because the play’s characters are so frequently uncertain of their own, and others’ identity.

Also lowered in the play is PersonProperty, the LAT which picks up familial roles (‘father’, ‘mother’, ‘sister’ etc) and social ones (job titles) – if you add this to the lowered rate of pronouns, then a rather strange social world starts to emerge, one lacking the normal points of orientation (and the play is also low on CommonAuthority, which picks up appeals to external structures of social authority – the law, God, and so on).

The visualisation, and Docuscope screens, provoked a discussion I found fascinating: we agreed that the action of the play seems to exist in an eternal present. There seems to be little sense of future or past (appropriately for a dream) – and this ties in with the relative absence of LATs coding for past tense and looking back. As the LATtice heatmap first indicated, MSND is unlike any of the recognised Shakespearean genres – but digging into the data shows that it is unlike them in different ways:

  • It is unlike comedy in its lack of features associated with verbal interaction
  • It is unlike tragedy in its lack of first person forms (though it is perhaps more like tragedy than any other genre)
  • It is unlike history in its lack of CommonAuthority

Waiting for my train back to Glasgow (at the excellent Euston Tap bar near Euston Station), I tried to summarize our findings in four tweets (read them from the bottom, up!):

 

 

I’ll try to keep in touch with the actors as they rehearse the play – this was a lesson for me in using the tools to spark an investigation into Shakespeare’s language, and I can now see that we could adapt these tools to various educational settings (including schools and rehearsal rooms!).

Jonathan Hope February 2012

Posted in Early Modern Drama, Shakespeare, Uncategorized | Tagged , , , , , | 4 Comments

What did Stanley Fish count, and when did he start counting it?

By Michael Witmore | Published: January 27, 2012

We have been observing the reaction to Stanley Fish’s critique of the Digital Humanities with great interest. Here is the full text of our comment, which could only be partially displayed on the New York Times comment window.

You know you’ve come up in the world if you’re being needled by Stanley Fish in The New York Times. Having done our share of work in the data mines, we believe Fish is right to insist that nothing in a text becomes evidence unless you have an interpretation which makes that evidence count. No amount of digital tabulation will substitute for a coherent, defensible reading.

As traditionally trained humanities scholars who use computers to study Shakespeare’s genres, we have pointed out repeatedly that nothing in literary studies will be settled by an algorithm or visualization, however seductively colorful. We have also argued that any pattern found through an iterative, computer-assisted analysis is meaningless without a larger interpretive framework in which to view it. It is the job of literary critics and historians to provide those interpretations, something they do by returning to the text and re-reading it with fresh eyes.

The job of digital tools is to draw our attention to evidence impossible or hard to see during normal reading, prompting us to ask new questions about our texts. This ability to redirect attention and pose new questions is the strong suit of certain kinds of digital humanities research. Indeed, we believe the addition of a digital prosthetic to our insistently human reading complements the skills of close textual analysis that are the staple of literary training. Not everyone in the so-called Digital Humanities community would agree with this position, but we believe the old and new techniques are entirely compatible.

What does it matter why Stanley Fish started minding his ps and bs in Milton? The point is that he has produced a plausible interpretation of Milton’s work based on evidence that fits his larger claim. The fact that an algorithm (“count ps and bs”) has directed his attention to something he hadn’t noticed doesn’t make the resulting pattern gibberish. You bet there are interesting patterns that show up in Milton when you mind his ps and bs. They existed before you counted them, and they exist after. However he found it, Fish has used that patterning to produce an interesting argument about the role of sound in Milton’s prose. And he has the evidence to back this argument up. In the end, he’s doing what most literary critics do in their work: create an interpretation that builds meaningfully on evidence in the text. Is there really any other way?

Yours sincerely,

Jonathan Hope, Strathclyde University

Michael Witmore, Folger Shakespeare Library

You can view a sample of our work at here.

Posted in Quant Theory | Tagged , | 3 Comments