Alan Liu
LightNews LightNews
banner
alanyliu.bsky.social
Alan Liu
@alanyliu.bsky.social
2.8K followers 300 following 310 posts
I'm a professor in digital humanities & English at UC Santa Barbara. For public humanities, I founded http://4Humanities.org & co-founded https://center-humanities-communication.org/. My website: https://liu.english.ucsb.edu/
Posts Replies Media Videos
alanyliu.bsky.social
Fan letter I wrote to journalist Alina Tugend for her “Building a Thriving Humanities Program,” Chronicle of Higher Education, Sept. 25, 2025, www.chronicle.com/report/licen.... (A non-paywalled PDF here: www.cla.purdue.edu/academic/cor... )
Screenshot of email sent by Alan Liu to Alina Tugend on Nov. 6, 2025. Text of the email reads: "Dear Alina, I finally had a chance to read your piece on "Building a Thriving Humanities Program." I just wanted to send you a note expressing my admiration of it. It's terrific longform reporting grounded on superb research from all over the nation. The piece is just bristling with reports on innovative programs and curricular revisions in the humanities elsewhere, but is framed in a way that brings a unifying perspective to it all so that the sum of the efforts seems like more than panicked humanities people throwing everything at the wall to see what sticks. All the tactics, in your writing up of them, suggest broader strategies. Kudos to you."
November 7, 2025 at 6:46 AM
alanyliu.bsky.social
A portrait of influence: my ridiculously over-annotated copy of John Guillory’s _Cultural Capital: The Problem of Literary Canon Formation_. I hadn’t looked at my physical copy for years but pulled it off the shelf today to recommend to a grad student a line of thought that John develops.
Top-down view of Alan Liu's hardback copy of John Guillory’s book _Cultural Capital: The Problem of Literary Canon Formation_ showing a dense mass of post-it notes marking annotated pages. Cover of hardback copy of John Guillory’s book _Cultural Capital: The Problem of Literary Canon Formation_ showing a dense mass of post-it notes marking annotated pages.
October 19, 2025 at 10:13 PM
alanyliu.bsky.social
Fellou cited 1 article for which I was lead author but collected many items from it, in the same way it collected 14 items from 1 of Geoffrey Harpham's pieces. My post discusses this at one point (see screenshot). Re: international balance, adjusting the prompt could change that, but I haven't tried
Screenshot of a paragraph from Alan Liu, “Humanities Definitions Research Project: An Experiment with Agentic AI,” 5 October 2025, https://liu.english.ucsb.edu/humanities-definitions-research-project-an-experiment-with-agentic-ai/. The paragraph contains the following text: "Thirdly, Fellou can concentrate on particular sources to mine multiple materials in bursts. For example, Fellou collected 14 separate paraphrases or quotes from Geoffrey Galt Harpham’s essay “Finding Ourselves: The Humanities as a Discipline.” This may or may not be a “feature,” rather than “bug,” depending on the needs of a researcher. But in any case it means that a researcher needs to take care to ask Fellou to scale its searching well beyond the quota of 100 found items that I asked for in my prototype test to ensure that the results are representative of breadth. Fellou could usefully give the human researcher a better sense of how it chooses to collect from sources. For instance, is it looking for relatively short, discrete passages that pass the threshold of a relevancy score?"
October 6, 2025 at 9:53 PM
alanyliu.bsky.social
My blog post reporting on testing the use of agentic AI (the Fellou.ai browser) to start a research project on gathering definitions of the humanities: “Humanities Definitions Research Project: An Experiment with Agentic AI” (liu.english.ucsb.edu/humanities-d...).
Screenshots of the Fellou interface while its browser agent is in progress on a research task prompted by the user. (A) and (B) represent windows with multiple browser tabs open to searches. Each tab in these windows shows rapidly changing thumbnails of the websites, articles (including PDFs), and other materials that Fellou is currently searching and analyzing. A panel at the right of each window shows a partial view of the kinds of sources and topics Fellou has planned to search in response to the user’s prompt. (C) is a window showing a periodically refreshed report on progress so far. In this window, the panel at the left shows in separate blocks some of the kinds of sources and topics that Fellou plans to search. The wide panel in the center summarizes results. And the panel at the right shows in blocks the specific sources and key phrases currently being “deep searched.”
October 6, 2025 at 12:07 AM
alanyliu.bsky.social
Here's the table of contents of The Humanities Laboratory: From Design to Archaeology. I recommend the editors’ introduction, “What Goes on in Humanities Laboratories?” (9-62). It’s truly wonderful: rich, deep, thoughtful.
Table of contents of The Humanities Laboratory: From Design to Archaeology: What Goes on in Humanities Laboratories? .............................................................9 Aleksandra Kil-Matlak, Jacek Małczyński, Dorota Wolska Part 1: Humanities Laboratories Chapter 1 The Tricky Case of Claude Lévi-Strauss’s Laboratoire d’Anthropologie Sociale................................................................................................65 Aleksandra Kil-Matlak, Jacek Małczyński, Dorota Wolska Chapter 2 The Laboratory as a “Soap Bubble”: Juri Lotman, the Tartu–Moscow School and the Laboratory of History and Semiotics.............................................87 Jacek Małczyński Chapter 3 Jerzy Grotowski’s Laboratory Theatre: A Laboratory in Search of the Truth................................................................................................... 139 Mirosław Kocur Chapter 4 An Archaeology of Humanities Infrastructure: Paul Otlet and Laboratorium Mundaneum..................................................................................... 175 Aleksandra Kil-Matlak Chapter 5 The Private Banker of Academia: The Case of Aby Warburg’s Laboratory of Cultural-Scientific Picture-History............................................... 213 Karolina Charewicz-Jakubowska 6 Contents Part 2: Laboratorisation of the Humanities Chapter 6 The Laboratory Effect: Laboratories in Leipzig at the Turn of the Twentieth Century................................................................................................... 239 Krzysztof Łukasiewicz Chapter 7 The Laboratorisation of the Field: The Formation of the Anthropological Mode of Knowledge............................................................. 281 Adam Pisarek Chapter 8 Jeremy Bentham and the Formation of the Laboratory of Power-Knowledge.................................................................................................... 317 Rafał Nahirny Part 3: The Post-Humanist Laboratory Chapter 9 T…
September 25, 2025 at 8:13 PM
alanyliu.bsky.social
Putting on my infrastructure studies hat to boost this just-published book: The Humanities Laboratory: From Design to Archaeology, edited by Aleksandra Kil-Matlak, Jacek Małczyński, & Dorota Wolska. www.peterlang.com/document/161...
Screenshot of Peter Lang publisher's web pat for The Humanities Laboratory: From Design to Archaeilogy. The page shows the book cover and a summary of the boojk. The summary reads: "This book explores the epistemic status and potential of humanities laboratories. It investigates the history of such laboratories, while contributing to debates on designing contemporary forms of labs. The book traces the trend for laboratories in the humanities since the mid-nineteenth century, outlining a multitude of projects across diverse times and spaces. We are interested in what makes humanities laboratories different from their scientific relatives. Can a humanities lab and those investigating natural sciences even be considered as related? Or should the relationship between them be seen transversally, outside of the opposition between the humanities and the natural and formal sciences? We argue that the humanities laboratory should not be based on the idea of mimesis and imitate a scientific lab, but rather operate according to the principle of mimicry. Only in this way can the humanities fulfil their self-critical function."
September 25, 2025 at 8:11 PM
alanyliu.bsky.social
“Liberal Arts Pantocrator” works into my argument short discussions of two early pictorial illustrations (by Herrad of Landsberg & Botticelli ) of the “seven liberal arts.”
Sandro Botticelli, A Young Man Being Introduced to the Seven Liberal Arts, 1483-1486 (fresco, detached and mounted on canvas). Public domain. The fresco shows the figure of Grammar (one of the seven liberal arts) leading by the hand a “lord” (likely Lorenzo Tornabuoni of a Florentine banker family connected to the ruling House of Medici) into the semi-circle of the other seven liberal arts.
September 24, 2025 at 8:00 PM
alanyliu.bsky.social
I made a short 3-page essay from part of my “Data Science & the Post-Liberal Arts University” in Critical Inquiry that had to be cut due to length: Liu, Alan. “Liberal Arts Pantocrator.” KCWorks, 24 Sept 2025, doi.org/10.17613/y9e...
Partial screenshot of first page of my "Liberal Arts Pantocrator" essay. The explanation at the top of the page reads: "The following fragment discussing two early pictorial illustrations of the liberal arts was originally part of my article on “Data Science and the Post-Liberal Arts University” (Critical Inquiry 51, no. 4 (2025): 597–618) but had to be cut due to length limitations. The fragment complements the article’s final section on “A Post-Liberal Arts Curriculum (2),” which discusses what is “needed for the post-liberal arts to enhance existing efforts in the liberal arts to be more inclusive of their social margins and global peripheries—in particular, by acknowledging the heritage of applied knowledge embedded in the very notion of such margins and peripheries” (612-618)."
September 24, 2025 at 7:56 PM
alanyliu.bsky.social
Just back from week as member of Santa Fe Institute’s working group on “The New New Science” with a group of mathematicians, computer scientists & (digital) humanists (www.santafe.edu/events/the-n...). Impressed by the quality of the conversations we had on math, humanities, modeling, and much else.
Screenshot of part of Santa Fe Institutute's web page announcing "The New New Science" working group event. The text shown on the page begins as follows (and is followed by the names of the working group organizers: Dan Rockmore, Renana Keydar, and Barak Sober): Thaw Library - Miller Campus Working Group All day September 15, 2025 – September 18, 2025 This working group brings together a small group of scholars with expertise spanning the humanities, mathematics, and computer science. Digital representations of knowledge and artistic corpora have opened new avenues for thinking about traditional Humanities subjects such as literature, linguistics, classics, and history and to rigorously examine mathematical and computational metaphors used in humanistic endeavors, while also exploring how a priori mathematical ideas can find meaning and epistemological value in humanistic subjects now considered natively digital. Dan Rockmore, an organizer of the Santa Fe Institute working group on "The New New Science," at a whiteboard in the meeting room of the SFI's Miller campus where the group met. Various humanistic and mathematical ideas are scrawled or listed on the board.
September 22, 2025 at 10:38 PM
alanyliu.bsky.social
CFP & flyer for 2026 Texas Tech U symposium on “AI & the Futures of the Human” where I will be a keynoter www.depts.ttu.edu/english/prog.... “How does the need to reconceive the humanities in the face of the rise of AI clarify, challenge, or transform our understanding of what it means to be human?”
Flyer for the 2026 Texas Tech U. Comparative Literature Sumposium on "Artificial Intelligence and the Futures of the Human" showing a view of the campus at the top and photos of the four keynote speakers at the bottom (Alan Liu, Kalindi Vora, Neda Atanasoski, Bruce Clarke). The flyer's abbreviated version of the symposium's call for papers reads as follow. (The QR code links to the full CFP.) <quote>~~New Generations, New Critical Inspirations~~ In the age of AI, what are the place of the “human” in the humanities and the role of the “humanities” in defining the human? How do we—and how do literature, cinema, and art—reconsider creativity, meaning, and culture in a world where machines can generate and purport to interpret texts, images, and ideas? What new ethical, philosophical, and pedagogical challenges arise when human and machine intelligence intersect? And how does the need to reconceive the humanities in the face of the rise of AI clarify, challenge, or transform our understanding of what it means to be human? This symposium brings together scholars, educators, artists, and technologists to reflect critically on the futures of humanistic inquiry and the futures of the human in the age of AI. We invite contributions that explore the opportunities and challenges AI presents for literature, history, philosophy, art, languages, and cultural studies. Please send your (1) 100-word abstract, (2) 50-word bio-info, and (3) questions to Dr. Y. Shu at eng.complit@ttu.edu by December 1, 2025 </quote>
September 22, 2025 at 8:40 PM
alanyliu.bsky.social
How research used to be! Pages from British newspapers during the French Revolution I once laboriously gathered from microfilm for my Wordsworth book by printing page parts to a photocopier & taping them together.
Scattered mass of phototopy pages (some with yellow Post-It notes, lying on a concrete surface. These are copies of pages from British newspapers & magazines covering the French Revolution (including London Times, Morning Chronicle, Morning Post, and Annual Register) that I harvested in the 1980s by printing page-parts from microfilm to a photocopier and then taping them together. I gathered these materials for my first monograph, Wordsworth: The Sense of History.
September 5, 2025 at 11:18 PM
alanyliu.bsky.social
The Academic Senate of UC Santa Barbara recognized me as its 2024-25 Faculty Research Lecturer, “the highest honor the UC Santa Barbara faculty can bestow on one of its members.” I was asked for language in response. This came pouring out of me: liu.english.ucsb.edu/alans-respon...
Screenshot of my email responding to my campus's Humanities & Fine Arts communications director asking for a response to being named Faculty Research Lecturer. The text of the response begins: ~~~~~~~~~~ Dear [communications director], here is some language about receiving the Research Faculty Lecturer award. This is too much for your purpose, I realize. Perhaps you can extract something from this? It’s a distinct honor to be recognized by my colleagues as the annual Faculty Research Lecturer for 2024-25, especially when I look back on the history of the award and realize what company I stand in. Just reviewing who the past Faculty Research Lecturers were from my own English Department makes me humble.† They include some of the most illustrious scholars in literary studies and — most moving to me and others — recently some of the dearest in the hearts of their colleagues for nurturing students and younger colleagues. Being recognized for research is especially meaningful to me now, given the direction my work has taken toward addressing the erosion in the public status of research expertise. My own research continues apace. Recently, I’ve focused on how data science, machine learning, and AI extend, but alter, the knowledges and practices that are foundational for a liberal arts education, which in the American educational system spans the sciences, social sciences, and humanities and arts.†† But my own research aside, the reason I say that recognition for research is so meaningful now is that the public perception of the value of research expertise has perhaps never been more challenged than today. That’s why, as I come to the end of my career, I’ve thrown myself into co-founding a new national organization called the Center for Humanities Communication....
July 2, 2025 at 9:27 PM
alanyliu.bsky.social
My “Data Science and the Post-Liberal Arts University” is just out in Critical Inquiry, www.journals.uchicago.edu/doi/10.1086/.... Thanks to the CI editors for acute feedback pushing me to enrichen the argument of this essay significantly.
Screenshot of the abstract for Liu, Alan. “Data Science and the Post-Liberal Arts University.” Critical Inquiry 51, no. 4 (2025): 597–618. (Also shown is an image of the cover of this issue of the journal, which is a clever media-archaeology style mock-up of an antique library card catalog with six drawers each labeled with the author and title of one of the articles in the journal issue.) The abstract for Alan Liu's article reads: "Data science has grown explosively in higher education, offering undergraduate degrees in the US on a “core and domains” curricular model that overlaps with—and in an intriguing way—replicates the multidisciplinary model of liberal arts education. This essay treats data science as a pathfinder for the continuing evolution of American higher education from the liberal arts to a post-liberal arts centered on applied knowledge, including today’s new modes of predictive/generative knowledge. The essay builds toward a consideration of how the post-liberal arts university can teach students both to apply knowledge and to know the meaning (historical, intellectual, and social) of application. A key for the post-liberal arts university will be to teach preprofessionalism in ways that do not just acknowledge the historical exclusion of the people of applied knowledge (at the social margins and global peripheries) from liberal arts education for the “free man” but turns such acknowledgement into new forms of liberal arts knowledge. Data science, the essay concludes, has the potential to contribute novel ways of conceptualizing intersectionality in general and in relation to applied knowledge. But data science is also constrained in this regard on the global scene where data power belongs to many regions and actors without a shared tradition of liberal arts education and thus a framework for a post-liberal arts carrying on shared ideas and practices of freedom."
June 26, 2025 at 7:47 PM
alanyliu.bsky.social
My article on “Data Science and the Post-Liberal Arts University” just came out in the summer 2025 issue of Critical Inquiry, criticalinquiry.uchicago.edu/current_issue/
Screenshot of the abstract for Liu, Alan. “Data Science and the Post-Liberal Arts University.” Critical Inquiry 51, no. 4 (2025): 597–618. The abstract reads: "Data science has grown explosively in higher education, offering undergraduate degrees in the US on a “core and domains” curricular model that overlaps with—and in an intriguing way—replicates the multidisciplinary model of liberal arts education. This essay treats data science as a pathfinder for the continuing evolution of American higher education from the liberal arts to a post-liberal arts centered on applied knowledge, including today’s new modes of predictive/generative knowledge. The essay builds toward a consideration of how the post-liberal arts university can teach students both to apply knowledge and to know the meaning (historical, intellectual, and social) of application. A key for the post-liberal arts university will be to teach preprofessionalism in ways that do not just acknowledge the historical exclusion of the people of applied knowledge (at the social margins and global peripheries) from liberal arts education for the “free man” but turns such acknowledgement into new forms of liberal arts knowledge. Data science, the essay concludes, has the potential to contribute novel ways of conceptualizing intersectionality in general and in relation to applied knowledge. But data science is also constrained in this regard on the global scene where data power belongs to many regions and actors without a shared tradition of liberal arts education and thus a framework for a post-liberal arts carrying on shared ideas and practices of freedom."
June 25, 2025 at 10:49 PM
alanyliu.bsky.social
New interview/post by Christine Henseler on our Center for Humanities Communication’s blog: “Johanna Drucker on Identification, Imagination, and Problem-solving to Communicate the Humanities.” center-humanities-communication.org/humcomm-talk...
Screenshot of the top of a blog post titled "Johanna Drucker on Identification, Imagination, and Problem-solving to Communicate the Humanities" on the Center for Humanities Communication site. The byline is "Christine Henseler, June 18, 2025). Four paragraphs at the beginning of the post are visible in the screenshot--as follows: <begin quote> Distinguished scholar, writer, and artist, Johanna Drucker recently shared her insights with me on designing effective humanities communication to engage undergraduate and high school students. Our conversation began with Drucker’s thought-provoking articulation of a central challenge facing humanities communicators: “We must shift from speaking about the humanities to enabling students to speak for and through them.” </end quote> Drucker has seen traditional humcomm methods all too often fail to captivate students’ imaginations and their evolving sense of identity, resulting in a disconnect between the subject matter and their lived experiences. The solution? “Find a point of entry,” she said, “one that fosters identification and imagination, prompting young people to actively inhabit scenarios, ask questions, and solve problems.”
June 19, 2025 at 10:34 PM
alanyliu.bsky.social
Scott Newstok today sent me a list of scholarship on machine learning/AI & “close reading” (and related materials) culled from his Close Reading Archive, with permission to share. I’ve put it up in a post: liu.english.ucsb.edu/machine-lear...
Screenshot of top of a post titled "Machine Learning & Close Reading Resources." A screenshot of a computer. The screenshot shows the following description at the top: "Scholarship on Machine Learning/AI and Close Reading culled by Scott Newstok from his Close Reading Archive (Scott Newstok shared this list with Alan Liu on May 18, 2025, with permission to share publicly.)" Then the screenshot shows the beginning of a list of 18 works with links.
May 18, 2025 at 10:41 PM
alanyliu.bsky.social
Learned from Scott Newstok of this study of large language models that evaluates their use for "1) close reading as a form of reasoning … 2) college-level knowledge in the literary domain & 3) figurative language understanding as ... reading comprehension” arxiv.org/abs/2505.09825
Screenshot of abstract on arXiv of pre-print by Peiqi Sui et al. on "KRISTEVA: Close Reading as a Novel Task for Benchmarking Interpretive Reasoning." The abstract reads: "Each year, tens of millions of essays are written and graded in college-level English courses. Students are asked to analyze literary and cultural texts through a process known as close reading, in which they gather textual details to formulate evidence-based arguments. Despite being viewed as a basis for critical thinking and widely adopted as a required element of university coursework, close reading has never been evaluated on large language models (LLMs), and multi-discipline benchmarks like MMLU do not include literature as a subject. To fill this gap, we present KRISTEVA, the first close reading benchmark for evaluating interpretive reasoning, consisting of 1331 multiple-choice questions adapted from classroom data. With KRISTEVA, we propose three progressively more difficult sets of tasks to approximate different elements of the close reading process, which we use to test how well LLMs may seem to understand and reason about literary works: 1) extracting stylistic features, 2) retrieving relevant contextual information from parametric knowledge, and 3) multi-hop reasoning between style and external contexts. Our baseline results find that, while state-of-the-art LLMs possess some college-level close reading competency (accuracy 49.7% - 69.7%), their performances still trail those of experienced human evaluators on 10 out of our 11 tasks."
May 17, 2025 at 6:33 PM
alanyliu.bsky.social
#LiuNetGuide1994, 8.3: “Imagine that our combined interface to the [information superhighway] is a windshield … [but] something like a recent jet fighter with ‘heads-up display’”
Diagrammatic visualization showing what in 1994 the Internet might look like if the metaphor of the "information superhighway" were instantiated in the form of looking through a "windshield." A windshield-like shape shows common interfaces/protocols for accessing the Internet at the time, including browsers like Mosaic or Lynx, email tools like PINE, file access tools like Gopher, and Usenet readers like NN. Still on the horizon at the top is the World Wide Web, just then coming into view.
May 14, 2025 at 5:48 PM
alanyliu.bsky.social
In my experience, an author's writings have a short half-life for their own author. I rarely read my publications after a year or so. Someone reminded me of my 1996 essay, “The New Historicism and the Work of Mourning.” Surprised by my earlier self. it's pretty good. www.jstor.org/stable/25601...
Screenshot of the opening of Alan Liu's essay, "The New Historicism and the Work of Mourning." The text in the screenshot is as follows: Imagine that a creature—let us leave it nameless, ageless, even specieless—drowns. The rope it had been clinging to slips away as the creature begins its slow descent to the tidal shelf twelve fathoms down. Here, on the pale bottom sands, the rough sea is only a massive but gentle surge. The body rocks back and forth on the sands, one arm half raised and adrift. Planking rains down nearby, one iron-bound section pinning the arm to the sand. Time passes and the soft parts of the body lose their definition, swell and break open bloodlessly against the planking and sand. The body becomes no longer quite a body; it is a body/plank/sand assemblage. Fish pick at the flesh and graze the algae on the plank. They become fish/body/algae assemblages. Shellfish, small crabs, other bottomfeeders join the ensemble. Microorganisms are at work, too, churning the mass’s internal structure and chemistry, fusing certain tissues more tightly, disassembling others and offering their elements up to the seawater as a subtle scent. . . . history considered universally is loss. History, as it were, is the perpetuation or retention of the process of loss. Thus listen to the distinctively elegiac note that sounds the largest theses of history in romantic New Historicism: “All discourse may well be about loss,” Levinson says. . . . (p. 559)At the end of six weeks, the body lies cradled in an encrusted, rooted, intricate matrix linked by thin trails of atoms (borne along tidal currents and fish migration paths) to a host of static and motile assemblages elsewhere—the total aggregate gradually propagating outward like a network of veins across the larger body that is the ocean. Nothing of the original creature has been lost: not an atom. But we know how to fathom this point of view because we recognize in it the makings of our modernity—or, more accurately,…
May 14, 2025 at 8:26 AM
alanyliu.bsky.social
I recommend strongly that department chairs & candidates involved in hiring, promotion, & tenure cases related to digital scholarship tune into this recording of a webinar on “Using the MLA Guidelines for Evaluating Digital Scholarship”: www.mla.org/About-Us/Gov.... Its content is quite substantial.
Title screen for MLA Professional Development Webinar of April 7th, 2025, on "Using the MLA Guidelines for Evaluating Digital Scholarship." The title screen lists speakers on the panel (Amy Earhart, Alan Liu, Roopika Risam, Sylvia Fernández Quintanilla) and the moderator (Nicky Agate).
May 2, 2025 at 7:44 PM
alanyliu.bsky.social
Video recording of the MLA’s “Using the MLA Guidelines for Evaluating Digital Scholarship” has been posted: www.mla.org/About-Us/Gov...
Title screen for MLA Professional Development Webinar of April 7th, 2025, on "Using the MLA Guidelines for Evaluating Digital Scholarship." The title screen lists speakers on the panel (Amy Earhart, Alan Liu, Roopika Risam, Sylvia Fernández Quintanilla) and the moderator (Nicky Agate).
May 1, 2025 at 8:46 PM
alanyliu.bsky.social
Blast from the past: In 1994 I wrote an “Ultrabasic Guide to the Internet -- For Humanities Users at UCSB.” 124 page. Self-published & sold through my university’s bookstore. Here’s a PDF: liu.english.ucsb.edu/wp-content/d.... I can’t believe the effort I put into this back then!
Cover of "Ultrabasic Guide to the Internet -- For Humanities Users at UCSB" showing the title followed by: "Version 1.0 (beta), By Alan Liu, English Dept."
April 28, 2025 at 6:02 PM
alanyliu.bsky.social
And here is part of ChatGPT 4o’s comparison of my two topic models based on multiple measures in their MALLET diagnostics.xml files, not just the coherence scores (partial screenshot). The comparison includes histogram charts.
Screenshot of histogram graphs with commentary produced by ChatGPT 4o when prompted for a comparison of the quality of two topics models of the same corpus based on the MALLET diagnostics file for each model. The commentary concludes: "These plots make it visually clear that the 100-topic model generally outperforms the 50-topic version across several core topic modeling quality measures."
April 13, 2025 at 9:22 PM
alanyliu.bsky.social
I’m continuing to experiment with asking AI to help interpret & evaluate topic models. Here’s a partial screenshot of results from asking ChatGPT 4o to compare the coherence of 2 topic models (of same corpus, but with different numbers of topics) based on their MALLET diagnostics.xml files.
Screenshot of part of the results when asking ChatGpt4o to evaluate and compare the coherence of two topic models (a 50-topic model and a 100-topic model of the same corpus) based on the diagnostics file produced by the MALLET program in creating topic models. The screenshot shows the following information: Model Average Coherence Best Topic Coherence Worst Topic Coherence 50topics_model -178.95 -47.89 -351.58 100topics_model -113.22 0.00 -260.56 Interpretation: 100topics_model is significantly better in coherence overall: Its average topic coherence is much higher (less negative) than the 50-topic model. The best topic in the 100-topic model has perfect coherence (0.00), which typically means its top words co-occur very strongly. Even the worst topic in the 100-topic model is more coherent than the worst in the 50-topic model. This suggests that increasing the number of topics from 50 to 100 helped MALLET identify more semantically cohesive topics. This is especially useful if your dataset is rich and varied. You can now browse the individual coherence scores for the 100-topic model in the table shown. Let me know if you’d like a visual comparison or further insights (e.g., distribution plot, filtering the best/worst topics, or assessing exclusivity). Screenshot of a graph with commentary produced by ChatGPT 4o when prompted for a visualization of the comparison of the coherence of two topics models of the same corpus. The commentary reads as follows: The boxplot above visually compares the topic coherence scores of the two models: 100topics_model shows a noticeably higher median coherence and a narrower range of low-coherence outliers compared to 50topics_model. This indicates that not only are the topics in the 100-topic model more coherent on average, but their coherence is also more consistently distributed. Let me know if you'd like to explore this further—perhaps by identifying and visualizing the top 5 most or least coherent topics from each model. ​
April 13, 2025 at 9:20 PM
alanyliu.bsky.social
Enjoyed joining the MLA’s webinar yesterday on using the new MLA Guidelines for Evaluating Digital Scholarship. I was asked to prepare an overview of our MLA Committee on Information Technology’s goals for the new guidelines (www.mla.org/About-Us/Gov...). Here are my notes for that (screenshot).
Screenshot of a page with Alan Liu's notes on "Main principles behind the revisions resulting in the new MLA “Guidelines for Evaluating Digital Scholarship." The first 5 of 7 principles on the page are: 1. Broaden the scope of the Modern Language Association “Guidelines on Evaluating Digital Scholarship” (substantially evolved from the MLA’s 2012 “Guidelines for Evaluating Work in Digital Humanities and Digital Media”) to recognize the degree to which digital scholarship is now profession-wide. 2. Provide more detailed guidance about documenting and evaluating digital scholarship to take account of its special features (e.g., evergreen work, collaborative work, public-facing work, digital technical and ethical standards). 3. Provide guidance on rewarding digital teaching and digital service, which in previous guidelines from the MLA and most other scholarly professional associations were rarely or thinly discussed. 4. Stress the importance of being inclusive of non-R1s (non-research universities) and also ”alt-ac” and “RSE” staff (people who are “alternative academics” and “research software engineers” hired into staff positions to support the digital humanities). Evaluation of digital scholarship occurs across a wide range of research and other higher-ed institutions; and applies across faculty and non-faculty in positions. 5. Shift the burden from individual digital scholars to their departments and institutions for establishing the expectations of what is required for promotion and tenure. Provide guidance to departments and universities about their responsibilities in: o educating their hiring or reviewing faculty about digital scholarship; o providing digital scholars with explicit guidelines and mentoring; o and supporting career progression for alt-ac and RSE scholars.
April 8, 2025 at 8:37 PM