Overall, GPT-5-Pro was good enough to use for my (informal) approach here—it was both internally consistent and looked good in accuracy spot checks. Its grades show some models scoring in the A- to A range, consistent with what others have found, too.

It turns out that some models are deeply inaccurate, and some are frequently inconsistent, but a few are reasonably consistent and accurate. And along the way, I learned that human graders are sometimes less consistent than we might hope.

I think this is one of the more common mistakes I see with people trying AI—the idea that if you go to a free chatbot, quickly run a question by it, and it does a bad job, then you've learned that AI cannot do a good job on that question.

The use of generated legal texts raises familiar issues in the world of AI (e.g. bias, inaccuracy) but also some distinct concerns (insincerity, a flood of documents). Check out our paper here: papers.ssrn.com/sol3/papers....

I've got a new paper up with the inimitable @kevintobia.bsky.social: "Generated Legal Texts"—about texts generated by AI and used in legal institutions. These texts are arising frequently in legal contexts around the world, perhaps faster than many realize. And, we argue ...

This blog post also contains my favorite AI+Law anecdote so far from the teaching-and-talks circuit.

I haven't spent much time with the new o3 model from OpenAI, but it is the first model to get all of the questions right on the informal testing of legal questions that I've been doing for a while when new models come out.

Several good things about this order. A small, technical detail that is good but easy to miss is the court's ruling on FRCP 65(c). In other times this would be unremarkable (or just absent from the order), but the Trump admin has tried to use Rule 65 to make this kind of civil action harder.

A mandatory arbitration agreement that requires subjecting disputes to an AI system. What could go wrong?

And here is a slightly different kind of problem, from another almost-a-real-case hypothetical involving copyright and disgorgement: 7/

And one from the consumer finance context involving a customer chatbot: 6/

I give a few examples in the paper from different contexts, and discuss a few different problems and implications. Here's a hypothetical from the antidiscrimination context, inspired by a real case involving a content moderation tool: 5/

In the growing world of AI litigation, class actions are going to be particularly important. In a new paper forthcoming in
the Wash. U. law review, I look at how class actions will influence the efficacy of AI regulations, and how they might even open up some new options for regulators. 🧵