{"id":567,"date":"2009-12-15T11:48:18","date_gmt":"2009-12-15T16:48:18","guid":{"rendered":"http:\/\/winedarksea.com\/?p=567"},"modified":"2025-02-10T17:53:56","modified_gmt":"2025-02-10T22:53:56","slug":"clustering-the-plays-without-principal-components","status":"publish","type":"post","link":"https:\/\/winedarksea.org\/?p=567","title":{"rendered":"Clustering the Plays Without Principal Components"},"content":{"rendered":"
\"Folio<\/a>
Folio plays clustered using all Language Action Types, Non-Standardized Data<\/figcaption><\/figure>\n

In comparison to the previous post, where we were using the plays’ scores on Principal Components to create clusters, here we are just using the percentage counts of the plays on all of the Language Action Types, the lowest level of aggregation in Docuscope’s taxonomy of words or strings of language. There are 101 Language Action Types or LATs, which is to say, buckets of words or strings of words that David Kaufer has classified as doing a certain kind of linguistic or rhetorical work in a text. I have made a table of examples of these types, taken from the George Eliot novel Middlemarch<\/em>, which can be downloaded here<\/a>.<\/p>\n

I find this diagram more than a little unnerving. It is quite accurate in terms of received genre judgments — notice that almost all of the Folio history plays (in green) are correct — and there are nice clusterings of both tragedies (tan) and comedies (red). Henry VIII<\/em>, which is here identified as a late play (blue), is placed in the cluster full of other late plays (including Coriolanus<\/em>, which could just as easily have been coded blue). And plays with a similar tone — Titus<\/em>, Lear<\/em>, and Timon<\/em> — are all grouped together as tragedies, separate from the other tragedies that are placed together further above. The strange pairing that repeats here from the Principal Component clusterings is Tempest<\/em> plus Romeo and Juliet<\/em>, something which merits further inquiry.<\/p>\n

Why should a mechanical algorithm looking at distances between counts of things produce a diagram this accurate? I’m not really sure. The procedure involves arraying each of the 36 plays in a multidimensional space depending on its percentage score on each of the things being rated here — the LAT categories. So, if “Motion” strings are one category, you can imagine an X axis with the scores of all the plays on “Motion,” with a Y axis rating all the plays on “Direct Address” as below:<\/p>\n

\"Direct<\/a>
Direct Address and Motion Scores in two Dimensions<\/figcaption><\/figure>\n

Now think about adding another score — First Person — to the third dimension, which will give us a spatial distribution of the plays and their scores on each of these three LATs:<\/p>\n

\"Direct<\/a>
Direct Address, First Person and Motion Scores of Folio Plays<\/figcaption><\/figure>\n

Now, there are distances between all of the points here and various methods (single linkage, complete linkage, Ward’s) for expressing the degree to which items arranged in such a space can be grouped together in a hierarchy of filiation or likeness. If you multiply out all of the things being scored in this analysis — that is, all 101 Language Action Types — you end up with a multidimensional space that is unvisualizable. But there are still distances among items in this multidimensional space, distances that can be placed into the algorithms for producing the hierarchy of likeness. That is what is going on — using Ward’s procedure with non-standardized data — in the visualization at the beginning of this post.<\/p>\n

As I’ve said before, a picture is nice, but just because you can reproduce a human classification with an algorithm doesn’t mean you’ve made any progress. You have to be able to show what’s going on in a text — which words are doing what things some or most of the time — before you can call your work an analysis. Perhaps that’s another reason why I find a diagram like this unnerving: I cannot work back from it to a passage in a text.<\/p>\n

By standardizing the data, we get the following re-arrangement. I am unsure how to categorize the benefits of data standardization in this case, but think this is a comparatively less compelling diagram:<\/p>\n

\"Clustering<\/a>
Clustering of Folio Plays using Standardized Data<\/figcaption><\/figure>\n","protected":false},"excerpt":{"rendered":"

In comparison to the previous post, where we were using the plays’ scores on Principal Components to create clusters, here we are just using the percentage counts of the plays on all of the Language Action Types, the lowest level of aggregation in Docuscope’s taxonomy of words or strings of language. There are 101 Language […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8],"tags":[37,38],"class_list":["post-567","post","type-post","status-publish","format-standard","hentry","category-shakespeare","tag-hierarchical-clustering","tag-standardization"],"_links":{"self":[{"href":"https:\/\/winedarksea.org\/index.php?rest_route=\/wp\/v2\/posts\/567","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/winedarksea.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/winedarksea.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/winedarksea.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/winedarksea.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=567"}],"version-history":[{"count":20,"href":"https:\/\/winedarksea.org\/index.php?rest_route=\/wp\/v2\/posts\/567\/revisions"}],"predecessor-version":[{"id":593,"href":"https:\/\/winedarksea.org\/index.php?rest_route=\/wp\/v2\/posts\/567\/revisions\/593"}],"wp:attachment":[{"href":"https:\/\/winedarksea.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=567"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/winedarksea.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=567"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/winedarksea.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=567"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}