Small Projects & Limited Datasets

I’ve been thinking a lot lately about the significance of small projects in an increasingly large-scale DH environment.  We seem almost inherently to know the value of “big data:” scale changes the name of the game.  Still, what about the smaller universes of projects with minimal budgets, fewer collaborators, and limited scopes, which also have large ambitions about what can be done using the digital resources we have on hand?  Rather than detracting from the import of big data projects, I, like Natalie Houston, am wondering what small projects offer the field and whether those potential outcomes are relevant and useful both in and of themselves as well as beneficial to large-scale projects, such as in fine-tuning initial results.

My project in its current iteration involves a limited dataset of about 4500 poems and challenges rudimentary assumptions about a particular genre of poetry called ekphrasis—poems regarding the visual arts.  It is the capstone project to a dissertation in which I use the methods of social network analysis to explore socially-inscribed relationships between visual and verbal media and in which the results of my analysis are rendered visually to demonstrate the versatility and flexibility available to female poets writing ekphrastic poetry. My MITH project concludes my dissertation by demonstrating that network analysis is one way of disrupting existing paradigms for understanding the social-signification of ekphrastic poetry, but there are more methods available through computational tools such as text modeling, word frequency analysis, and classification that might also be useful.

To this end, I’ve begun by asking three modest questions about ekphrastic poetry using a machine learning application called MALLET:

1.) Could a computer learn to differentiate between ekphrastic poems by male and female poets?  In “Ekphrasis and the Other,” W.J.T. Mitchell argues that were we to read ekphrastic poems by women as opposed to ekphrastic poetry by men, that we might find a very different relationship between the active, speaking poetic voice and the passive, silent work of art—a dynamic which informs our primary understanding of how ekphrastic poetry operates.  Were this true and were the difference to occur within recurring topics and language use, a computer might be trained to recognize patterns more likely to co-occur in poetry by men or by women.

2.) Will topic modeling of ekphrastic texts pick out “stillness” as one of the most common topics in the genre?  Much of the definition of ekphrasis revolves around the language of stillness: poetic texts, it has been argued, contemplate the stillness and muteness of the image with which it is engaged.  Stillness, metaphorically linked to muteness, breathlessness, and death, provides one of the most powerful rationales for an understanding how words and images relate to one another within the ut pictura poesis tradition—usually seen as an hostile encounter between rival forms of representation.  The argument to this point has been made largely on critical interpretations enacted through close readings of a limited number of texts.  Would a computer designed to recognize co-occurrences of words and assign those words to a “topic” based on the probability they would occur together also reveal a similar affiliation between stillness and death, muteness, even femininity?

3.) Would a computer be able to ascertain stylistic and semantic differences between ekphrastic and non-ekphrastic texts and reliably classify them according to whether or not the subject of the poem is an aesthetic object or not?  We tend to believe that there are no real differences between how we describe the natural world as opposed to how we describe visual representations of the natural world.  We base this assumption on human, interpretive, close readings of  poetic texts; however, there is the potential that a computer might recognize subtle differences as statistically significant when considering hundreds of poems at a time.  If a classification program such as Mallet could reliably categorize texts according to ekphrastic and non-ekphrastic, it is possible that we have missed something along the way.

In general, these are small questions constructed in such a way that there is a reasonable likelihood that we may get useful results.  (I purposefully choose the word results instead of answers, because none of these would be answers.  Instead the result of each study is designed to turn critics back to the texts with new questions.)  And yet, how do we distinguish between useful results and something else?  How do we know if it worked?  Lots of money is spent trying to answer this question about big data, but what about these small and mid-sized data sets?  Is there a threshold for how much data we need to be accurate and trustworthy?  Can we actually develop standards for how much data we need to ask particular kinds of humanities questions to make relevant discoveries?  In part, my project also addresses these questions, because otherwise, I can’t make convincing arguments about the humanities questions I’m asking.

Small projects (even mid-sized projects with mid-sized datasets) offer the promise of richly encoded data that can be tested, reorganized, and applied flexibly to a variety of contexts without potentially becoming the entirety of a project director’s career.  The space between close, highly-supervised readings and distant, unsupervised analysis remains wide open as a field of study, and yet its potential value as a manageable, not wholly consuming, and reproducible option make it worth seriously considering.  What exactly can be accomplished by small and mid-scale projects is largely unknown, but it may well be that small and mid-sized projects are where many scholars will find the most satisfying and useful results.

Curating a Network of Wood and Would

In my current research, I argue that Elizabeth Bishop’s poem “The Monument” represents a more democratic attitude toward aesthetic objects than what we see from her contemporaries Robert Lowell and John Berryman.  Eschewing the “tutelary” relationship between poet as teacher and reader as student, Bishop offsets her own position of power as the artist-creator by including the voice of a resistant, reluctant onlooker whose interrogations about what it is they are supposed to be looking at position the reader as the monument’s curator, one who must select between descriptions, views, depth, and purposes for the monument.  The monument’s physical presence is brought into being collaboratively between the two speakers who create it’s shape and it’s potential and the reader who must parse and prioritize the verbal network of the poem.  Furthermore, Bishop, who started writing “The Monument” in her Key West notebooks (see Barbara Page) after just having read Wallace Stevens’ Owl’s Clover, enters into public discourse about the relevance of public monuments (and by extension art) in a social, political, and economic climate in which people are suffering, nations are warring, and the realities of daily life seem to negate the place and purpose of art.  Sounds familiar, no?

Bishop responds by arguing that monuments (and painting, and poetry, and sculpture) are significant because they are sites of “commemoration”–an interesting word choice because the word “commemorate” requires communal activity.  Unlike her friend Robert Lowell who uses the bronze relief by August Saint-Gaudens in Boston Commons to establish historical connection and significance for himself as an artist, Bishop imagines the monument as evolving to purpose rather deriving from it.

Visualizing the poem as a network demonstrates how the speakers’ relationship to one another builds out of their discursive description of the monument.  In the networks below, each speaker is related to the monument through questions and description (recorded as the statements they make in the poem).  I have characterized those statements as grounding the monument in physical space, with tangible attributes, insisting on its materiality [repesented by blue lines] or on the other hand imagining the monument’s potential or possibility through questions, equivalences (is it this or that statements) and statements that intimate a metaphysical presence for the monument [in orange].

 

The networks are formed by matching each speaker up with with each statement made in the poem.  As a result, this is a “bimodal” network: one which takes actors and text and studies the relationship between them.  The relationship, represented by a line is further characterized as advancing the monument’s status as “wood” (a physical object belonging to the material, and therefore “real” world) and the word’s homophone “would” (a representation of possibility, potential, and by association the “imaginary” life of the mind).

The reader as curator, then, must choose among the wood and the would–between the multiply rendered descriptions of the monument’s physical presence and its imagined potential, which is a new beginning itself, the shape of which could be poetry, painting, statue or monument, depending on choices the reader makes.

Monument as Descriptive Network

Chasing the Great Data Whale

The first thing you hear, or at least that you should hear, when you present an idea for a digital humanities project to someone already familiar with the field is this: “That’s great! [pause]  What does your data set look like?”  Actually, that’s the reaction you’ll get if whoever you’re talking to is taking you seriously, so the reaction is a mixed blessing.  On the one hand, you have their attention.  On the other, they know enough to point out that projects without data go nowhere, and good data (not necessarily synonomous with big data) is hard to find. Data is truly the Moby Dick of the digital humanities.  None is ever big enough, clean enough, or well-structured enough to achieve precisely what it is that researchers would like to achieve.  Just when you (and more likely your team) feel confident that the data set is “harpooned” (structured, refined, aptly-tagged, and curated), the whale takes off again with a new standard: interoperabilty.  It doesn’t play well with other data, and the chase begins again.  When your data set works for you, that has some value, but when your data set works with others, well, that means a wider audience and a broader impact for your work.

Most projects have lifespans determined by fellowships or grants or sabbaticals, and unlike Captain Ahab, we can’t afford to have our hard work dragged out into the abyss and lost.  The DH mantra may well be “project or perish.”  Hard decisions about data are often determined by two factors: intellectual value and time.   First, your data needs to be thoughtfully selected, described (tagged with metadata) and clean enough in order to work and to reasonably make the argument that what you’ve done with the data can be trusted.  However, you need to know when it’s time to cut the rope and release what might be done, a choice between good-enough and great.  When just a few more hours of tagging, a few more weeks of correcting OCR errors seem just within your grasp, the choice feels mercinary.  Deciding to stop improving your data, though, is like the difference between Faulkner and Melville: “you must kill all your darlings.”  Data sets, really, are Modernist objects: they are “what will suffice.”

This is the lesson I learned during my first full month at MITH.  As a Winnemore Dissertation Fellow, I have approximately four months to capitalize on MITH’s resources and to produce a project that has a strong (but not perfect) data set, answers relevant humanities questions, and possesses enough sustainability, public outreach, and external support to become a viable, fundable project in the future.  In these first five weeks, I have benefitted from the wealth of experience and depth of knowledge that the team assigned to my project possess.  Jennifer Guiliano knows how to pull projects together, shepherding them through the steps of data management, publicity, and sustainability.  Taking to heart her sage wisdom about managing successful DH projects, I feel that I have a much stronger grasp on what steps must be taken now and what can and must happen later, professional development knowledge that more graduate students should have when they venture into the alt-ac or tenure track market. Trevor Muñoz’s expertise with data sustainability prompted questions that have helped shape a future to my project even at it’s earliest oustet—few graduate students have the time or the resources to think about how adequate curation in the short term could mean greater benefit in the long term.  Amanda Visconti and Kristin Keister have been helping me to shape a public portal for my work, and I know that the virtual presence they help create will lend to the future success of the project, as well as its intellectual value.

The most salient lessons I’ve learned about the lure of the great data whale, however, have been from Travis Brown.  I arrived in January with a large, but unrefined dataset of approximately 4,500 poems in a less-than-uniform collection of spreadsheets.  I had a basic understanding of what I wanted the data to look like, but in consultation, Travis helped me to determine what would work better and might last longer.  Travis created a data repository on MITH’s space with a version control system called “git.”  (Julie Meloni’s often-cited and useful “Gentle Introduction to Version Control” on ProfHacker provides a useful explanation of what git is, why it’s valuable, and where to find resources if you’d like to try it.)  Once I installed git on my machine, replicated the repository, and “pushed” (basically moved the data from my computer into the repository Travis created) the data to it, Travis could take a look.  We agreed to separate the text of the poems and their titles from the descriptive information about it (author, date, publication, etc.) and to use a uniform identification number to name the file for each poem, and to track its descriptive data in a separate spreadsheet.  We realized at that point that there were some duplicates, and in conversation agreed that we would keep the duplicates (in case there was a reason for them, such as minor textual differences) and tag them, so that later we could come back to examine them, but in the meantime not include them in the tests I’d like to run.

Then came the “darling killing.”  Metadata, the information that describes the poetic texts that make up my data set, is necessary for the tests I’d like to run—those that classify and sort texts based on the likelihood that words co-appear in particular poems.  The amount of metadata that I include to describe the poems will determine the kinds of tests and the reliability of the tests I hope to run.  However, tagging 4,500 poems, even with simple identifiers, could take the whole four months of my fellowship if I let it.  The hard choice was this: I would tag only the gender of the poet associated with each poem and whether or not the poem is ekphrastic (that is to say written about, to, or for work of visual art) or not or unknown.  Some poems were easily tagged as ekphrastic, because I had sought them out specifically for that purpose; however, more often than not, I would need to make poem by poem determinations about the likely poetic subject.  This takes time, and because of the tests I need to run to ask the questions I want to ask (eg. Can Mallet distinguish between descriptions of art and descriptions of nature?), I will need to let go of other useful, informative, helpful tags I would like to have, like the date each poem was published, the artwork to which it refers, and so on.

I am keeping record of all these things.  The decision not to be perfect is the right choice, but it isn’t an easy one.  I feel sometimes as though I have watched my whale slip from my grasp and sink back into the sea.  My crew is safe.  My project will continue on schedule, but not without the longing, nagging feeling that I need to return to this place again.  Perfect data sets are a myth, one that often forms the barrier to new scholars who wish to begin DH projects.  Rather than struggling for the perfect data set, I want to suggest that we place a much stronger emphasis on the more intellectual and more necessary component of data curation and data versions.  I would argue that we judge projects not by the “completeness” or “perfection” of the data, but how well its versioning has been documented, how thoroughly curatorial decisions such as what to tag, when, and why have been publicized and described, and how much the evolution of the data contributes to the development of other projects within the community.  Much the same way that we know more about the value of an article by how often it has been cited, we should value a digital humanities project by how much can be learned by the projects that follow it.  In this sense, we should all be called Ishmael.

Preparing texts for network visualization

When I presented at MSA 13 earlier this month, I was unsatisfied with my methods for creating network visualizations of texts.  I knew that preprocessing automatically would not work yet, since I have yet to identify precisely how I want to designate nodes across larger bodies of poems.  What I’ve been looking for is a way to mark texts up descriptively, using some form of markup language (XML, TEI), that would be uniform enough to render data that could be meaningfully displayed, and then to find a visualization software package with an algorithm that would “work” the way I wanted it to.  The problem, of course, is that when you’re a rogue DH scholar out in the world borrowing tools and using whatever tends to fall your way, then you’re not going to be sure about how each tool works (unless you have a CS or social science degree that includes learning about network algorithms, which I do not have), and this is going to detract from the validity of how and what you say about your object of study.  On the flip side, tools and text analysis software are becoming more widely available, and so doing what I’ve done, which is to say Googled “discourse network tool” and finding Philip Leifield’s “Discourse Network Analyzer” is actually possible.  What is remarkable about how DNA, a GUI text processing software, works is that it is designed as an interpretive tool to mark texts up in XML so that they can be displayed using free network visualizing software such as Visone, Ucinet, or Netdraw.  The designed purpose of Leifield’s DNA software is to collect articles on a topic area and to use those articles to create network visualizations of agreement and disagreement between individuals and groups.  For example, the sample dataset used for a tutorial on the software comes from someone at the University of Maryland named Dana R. Fischer, (I have no idea who she is… but I’m definitely going to look her up!) who marked up articles, testimony, and other texts about climate change.  Essentially, she could input each text into the DNA software and create a basic XML document with very minimal encoding (document type, author, dates, title) and then use DMA to select portions of text that create a “statement” about climate change.  By tagging the speaker, the organization the speaker is affiliated with, and the content type –a restricted list of terms created by the user to describe the topic being discussed—as well as whether or not the speaker agreed or disagreed with the topic) she could create networks of statements made about climate change that also included the individuals involved in the climate change debate and their organizations.  Such a visualization helps us to understand how much any one group (say, the Senate and the EPA) agree with one another, to identify the issues on which they agree and disagree, and to also understand affiliations (which speakers are affiliated with which climate change debates).

This isn’t *exactly* what I had in mind, but it’s really darn close.  The power of this particular piece of software is that I can be in charge of what constitutes an article (a poem), what constitutes a speaker (the poetic speaker, the author, the third person omniscient… all of them), and the “content” to be described.  Granted the “organization” classification is less helpful to me, but in the instance of “The Venus Hottentot (1825)” I could differentiate between speakers from the first section of the poem from the second using this feature.  Using the software this way does not begin to utilize it’s real power, which is to read topics and speakers over large corpuses of texts in similar ways.  For now, I’m looking at one poem; however, I could see in the future were I to take this poem and situate it in a larger public discourse about black female subjectivity, I could.  I could import, for example, Sander Gilman’s article “Black Bodies, White Bodies: Toward an Iconography of Female Sexuality in Late Nineteenth-Century Art, Medicine, and Literature,” which we know Elizabeth Alexander read before writing the poem.  We could also bring in articles by Sadiah Quershi on “Displaying Sara Baartman” or Terri Francis’s “I and I: Elizabeth Alexander’s Collective First-Person Voice, the Witness and the Lure of Amnesia,” or chapters from Deborah Willis’s Hottentot Venus 2010 and demonstrate how Alexander’s poem participates in a larger act of social recovery.

There are, as with any tool, limitations, though.  So far, the only way to create the visualizations is using the speakers, organizations, and categories with directional lines indicating agreement or disagreement.  I have not found a way of creating networks of “statements.”  In other words, I have not found a way to pull a category and then visualize the network of statements about that category and how they relate to each speaker; however, I have only begun the process of creating visualizations.  Another complication is that I have only found ways to make a statement associated with one category.  I’m fairly certain I can find a work around for that, but for the moment, that’s not worked out; however, I will say that having to choose between regular category designations (ones of my own creation) did make me very attuned to my assumptions about the text.  That process helped me to realize how my visualizations of these networks will always be limited and remind me that I need to make those limitations transparent when I write about what the visualization actually visualizes.

In the meantime, even though I am not teaching right now, I’m really excited about what this kind of software could mean for my students.  In the English 101 courses at the University of Maryland, students write three linked assignment papers on a self-selected research topics.  These are position papers, where the student must make purposeful arguments for what he or she believes in and respond to the discourse of the field in which their selected debate is ongoing.  We generally assign an annotated bibliography as the first part of that linked assignment as a way of getting students to read the work and to then explain who agrees with each other on particular points and who disagrees.  The hard part of this assignment is that each entry is generally 2 paragraphs long and includes only 8-10 sources, and getting the students to actually compare arguments, identifying points of agreement and disagreement is difficult.  However, if the assignment were to use the Discourse Network Analyzer to import each article and then go through each article tagging “statements,” “speakers,” “organizations,” and “categories” (for example, are the speakers arguing that a particular action should be taken or that one event causes another…) as well as “agreement” or “disagreement” with that statement, they might begin to see how their readings create a network of ideas and by understanding who agrees and what they agree upon, the student might be better able to situate him or herself within the discourse of that issue.  It’s an intriguing idea to me, and at some point when I’m teaching again, I think I’m going to make use of this technology.

Networks and Network Analysis for the Humanities: Reunion Conference

On October 20-22, the UCLA Institute for Pure & Applied Mathematics will host the “Networks and Network Analysis for the Humanities: Reunion Conference.”  Participants were 2010 NEH summer institute participants and they will present their projects during this 2 day event.  Coincidentally, Robert Nelson, this week’s MITH digital dialogues guest speaker will be one of the presenters at the IPAM event.

Looking at this list of projects and preparing to discuss network analysis at MSA13 this weekend prompted me to make a list of known projects using (social) network analysis to pursue questions in the humanities.  I’ve created a page where I attempt such a thing, and will post it on this site soon.