Promotion and Tenure Criteria for Assessing Digital Research in the Humanities

Digital scholarship in the humanities may be defined broadly as humanistic inquiry conducted through technological or computational means. Scholars in digital humanities work in many disciplines, and their research is often interdisciplinary and collaborative. These scholars frequently explore research questions that cannot be adequately treated via traditional methods. Digital scholarship in the humanities at times has affinities with applied Computer Science—for example, in the development of scholarly tools, encoding, programming and data mining of large corpora. Scholars in digital humanities often work with technologies that are in constant change.

An increasing number of professional associations have developed guidelines for the evaluation of digital humanities tenure or promotion files and have offered recommendations to faculty about how to document their scholarship as they create tenure and promotion folders. Among such resources are:

• The College Art Association and the Society of Architectural Historians have developed “Guidelines for the Evaluation of Digital Scholarship in Art and Architectural History” (January 2016) with funding from the Andrew W. Mellon Foundation. This is an excellent document with considerable granularity and relevance that goes well beyond the particular disciplines for which it was written.
• The Modern Language Association has produced “Guidelines for Evaluating Work in Digital Humanities and Digital Media” (2012).
• The American Historical Association has published draft “Guidelines for the Professional Evaluation of Digital Scholarship in History” (2015). The American Historical Association Working Group on Evaluating Public History also released a report, “Tenure, Promotion, and the Publicly Engaged Academic Historian: A Report” (September 2010), that addresses collaborative scholarship, including interdisciplinary and digital history projects, and provides recommendations to Department Chairs and Tenure and Promotion Committees.

Other professional associations are in the process of preparing similar statements, including the Society of American Archaeologists. All of the existing guidelines are highly recommended reading for committees, departments and colleges. The guidelines also may be useful for faculty preparing their folders.

Since the mid-1990s, University of Nebraska-Lincoln has recognized that digital technologies and the internet have changed the way we read, study and conduct research. Despite this now fairly long history, digital scholarship in the humanities still may be unfamiliar to some colleagues. Four points need to be highlighted:

• All of the professional association guidelines indicate that faculty engaged in digital humanities scholarship need to be evaluated both rigorously and fairly. Such evaluation includes engaging qualified reviewers who are practiced in the interpretation and development of new forms of scholarship and who are knowledgeable in a faculty member’s field.
• Work should be viewed in the form in which it is created since born-digital and web-based projects are often spatial, interactive, and networked.
• Digital scholars often rely upon research teams, much like scientists and some social scientists. Teams may be comprised of other scholars, librarians, archivists, technology experts and students. Consequently, tenure and promotion candidates must communicate the collaborative nature of the work to the tenure and promotion committees and must make clear their individual contributions to the team enterprise.
• Understanding the nature of digital humanities scholarship is all the more important at a time of uncertainty for the monograph. Print runs for many monographs have declined significantly and some presses are turning largely to print-on-demand. As university presses face the future with fewer resources, alternative forms of publication become more important. Digital humanities may offer an alternative means of publication with the potential to provide data, documents, and even images that cannot be included in a print monograph—and perhaps not even in an e-version of a typical monograph. It is vital, then, that digital publication or scholarship is affirmed when it meets the same standards of rigor and significance as would be required of print.

The traditional criteria of excellence in scholarship, impact, originality, and reputable publication apply to digital work in the humanities. The candidate's folders should provide clear documentation of the ways in which their scholarship qualifies with respect to content, process, and outcomes. Specifically, how does the digital nature of the humanities research contribute to its originality and excellence? What are the implications in terms of audience, pedagogy, and the creation of research tools? Likewise, promotion and tenure committees should review folders with these questions in mind.

The following are examples of criteria for evaluation that may be applied toward evaluation of candidates' folders. Not all will apply to each candidate; thus, committees will wish to use those most suitable to the candidate's research and discipline.

• Peer review of competitive national grant applications, published digital research sites and digital humanities tools. From the CAA/SAH guidelines: “External funding is critical to the realization of many digital projects and funding is highly competitive. Applications require that scholars secure multiple letters of support…Letters and application materials are reviewed by experts who frequently provide written feedback.” External peer reviews of digital scholarship and tools are increasingly available and when possible included in the candidate’s folders.
• Impact as evidenced through citations in other scholars’ works; unique users coming to the site; links from other reputable sites, etc. An assistant professor may have greater difficulty in demonstrating quantitative measures of impact than an associate professor, however, there should be some meaningful impact at any rank.
• Use and application of internationally accepted standards (i.e., Text Encoding Initiative guidelines, or other accepted international standards such as Encoded Archival Description [EAD], Resource Description Framework [RDF], Linked Open Data [LOD] and others). Metadata needs to be appropriate and complete, and conditions of use should be defined.
• Collaboration or connections with related digital research projects at other institutions.
• Technical innovation and sophistication of projects (i.e. LiDAR, 3D, GIS, Photogrammetry, multispectral imaging, etc.)
• Use of best practices in design and implementation, potentially in consultation with other experts in the field.
• Applying best practices for sustainability and long-term accessibility.
• Compatibility between design, content, and medium.

Additional Criteria

• Pedagogical application and assessment.
• Conference presentations.
• Print publications resulting from the digital research.
• Online databases resulting from the digital research. For example, in archaeology, online databases or archives of archaeological materials offer free access to materials and knowledge. Because of the datasets are so large-scale, they cannot be fully published in print.
• The re-usability of the archives.

In addition to the professional association guidelines, the following links may be useful to reviewers who need more information about digital humanities:

• Alliance of Digital Humanities Organizations: http://adho.org/
• A New Companion to Digital Humanities, 2nd Edition, ed. Susan Schreibman, Ray Siemens, John Unsworth. Wiley-Blackwell, January 2016.
• Companion to Digital Literary Studies, ed. Siemens and Schreibman: http://www.digitalhumanities.org/companionDLS/ - Last update 2008
• Text Encoding Initiative (TEI): http://www.tei-c.org/index.xml
• ACLS Cyberinfrastructure Report: http://www.acls.org/cyberingrastructure/
• ACH Guidelines for Assessment of Digital Scholarship in Tenure and Promotion: http://ach.org/ach-guidelines-for-assessment-of-digital-scholarship-in-tenure-and-promotion/

This document will continue to evolve as professional associations step forward with guidelines. In the meantime, we hope this statement provides some ideas that may help departments and colleges shape their criteria in meaningful ways. It is our hope that this document and the guidelines from professional organizations will aid in conversations at the departmental and college levels.

Thanks to Stephen Wheatley, American Council of Learned Societies, and Geoffrey Rockwell, University of Alberta for their contributions to the document.