I then focus on AI as tools for data preparation, using oral history transcriptions and OCR correction as case studies. In both domains AI models show promise in languages and content well-represented in their training data. But for data that falls outside of their training, LLMs often falter.
6/8

One approach to assessing LLM performance is on benchmarks of historical knowledge. In some domains LLMs now match “expert” level knowledge in tests like A.P. history, per the MMLU benchmarks. However, such performance may owe less to true historical fluency than "benchmark leakage."

4/8

Enjoying a pleasant evening at NC State University after giving a history lecture. Travel, talking about my nerdy interests, and meeting interesting new people - some days this job is quite nice.

Scenes from #NoKings in Belmont, NC.

I may not be the best foreign language learner, but I managed to keep this Duolingo streak going, which began during the pandemic. One thousand days seemed like an impossible stretch back then. Here‘s to persistence, even (maybe especially) when big things are feel beyond your reach.

That cancelled TV show you‘ll never stop thinking about?

This app is a work in progress, and while designed to provide factual responses, hallucinations do occur. This project is meant to critically highlight both the strengths and limitations of this technology.

Nicolay allows users to explore some of Lincoln's most important speeches via large language models and machine learning techniques like text embeddings and semantic search. You ask a question, and the app searches through the corpus and provide a response informed by the search results.

Greetings new followers! I'm a digital humanist working on historical applications of AI. Here's my a recent project, Nicolay:

nicolay-honestabes-info.streamlit.app

It explores how LLMs can be used to explore text collections, in this case Abe Lincoln's speeches. More below, thanks for following!

Data visualization of existential dread. With #midjourney.

As we designed this illustrated guide, we took stock of both generative AI's capacities and limitations. Discussion centered on the potential of this tech to empower creation and exploration, but also to distort our understanding of the past.

For example, the section on the scriptorium and library features not only AI imagery but also illuminated medieval manuscripts held by the Abbot Vincent Taylor Library at
Belmont Abbey College, allowing students to explore the history of medieval book production.

This classroom experiment asks the question: what might this monastery have looked like? We used multimodal AI to explore this speculative landscape. Historical commentaries were used to prompt GPT-4V and Midjourney to create an illustrated narrative guide to the plan.

The St. Gall Plan is a detailed architectural plan of a Benedictine monastery from the ninth century. It includes a basilica, library, brewery, infirmary, and many other structures. While never actually built, it represents an idealized depiction of medieval monasticism.

Pleased to share a generative AI project featured by the MLA-CCCC Joint Task Force on Writing and AI - Visualizing the St. Gall Plan. Below is a thread on how multimodal AI offers novel pedagogical opportunities:

#twitterstorians #generativeai #AIineducation #AIliteracy