Check out our paper for many more details: arxiv.org/abs/2505.12533
This work was done with my wonderful collaborators: Terra Blevins, Allan Hanbury, Michael Wiegand, and Gábor Recski.

#3: Structural issues in RE benchmarks, such as single-relation per sample constraints, reliance on external factual knowledge, and inconsistent negative class definitions, further impede model transferability. [5/5]

#2: Adaptation strategy is dataset-dependent. Fine-tuning works best with manually annotated data, while few-shot ICL offers advantages with noisier data. However, zero-shot baselines sometimes outperform all cross-dataset approaches! [4/5]

#1: RE models struggle with generalisation even within similar domains. Models that perform well on their training distribution often overfit to dataset-specific artefacts rather than learning robust relational patterns. [3/5]

TL;DR: We examine the OOD performance of RE models focusing on biographical relation overlap across general-purpose RE datasets and find that data quality dictates adaptation strategy: fine-tuning excels with clean data, ICL with noisy data. Zero-shot sometimes outperforms all! [2/5]