Oops, here are the three caused-motion examples that were meant to go in the first post:

👥Joint work with Abdullatif Köksal and Hinrich Schütze

📰Check out the paper: nejlt.ep.liu.se/article/view...

💻The full dataset and code are available on GitHub: github.com/LeonieWeissw...

🧵7/7

We find that a range of models indeed struggle with this, but Gemma 27B solves it almost perfectly!

In grey are cases where the model struggles to answer both questions, in red the cases where it would have needed to reply on the caused-motion semantics.

🧵6/7

Once we have manually annotated the final dataset in this way, we return to the original question and test if LLMs find the third question below easier than the second, which would indicate difficulties in making use of the semantics of the caused-motion construction.

🧵5/7

Designing the right prompt is tricky and depends on annotation cost.

For example, giving examples and asking for a json with the sentences and labels is always a good idea, but using o1 over 4o-mini is only worth it if human annotation costs more than .5$ per sentence!

🧵4/7

But this leaves many FPs, and filtering them by hand would be expensive.

To reduce this cost, we use few-shot prompt-based filtering, which greatly reduces the number of FPs that our human annotator will have to sift through, and therefore the annotation cost.

🧵3/7

They are all instances of the so-called caused-motion construction, and collecting enough instances for testing was a challenge, given its rarity!

To construct our dataset, we first create a dependency filter based on the syntactic side of the construction.

🧵2/7

There's many directions where this could go, multilingual, low-resource language, interpretability, depending on your profile, and the internship may lead to a PhD, provided we get funding!

As we found in aclanthology.org/2025.coling-... that BPE-based LLMs (i.e. pretty much all LLMs) did not handle prefixations well

Basically the idea is to extend www.pnas.org/doi/10.1073/... to see how well LLMs model competition between affixes, not only suffixes (e.g. -ity vs. -ness) but also prefixes (e.g. un- vs. non-)

This is great! Do you happen to also have a list of nice examples of lab websites? 🙂

@kanishka.bsky.social congratulations on making the list!!

Session 5: "BabyLM’s First Constructions." In a companion paper to the first one, we show that much of this constructional knowledge is even present in "babyLMs" with more developmentally plausible amounts of training: arxiv.org/abs/2506.02147. w/ @weissweiler.bsky.social

Session 4: "Constructions are Revealed in Word Distributions." We show that rich knowledge of syntactic constructions in masked language models (MLMs) is revealed by patterns of contextual constraint. arxiv.org/abs/2503.06048. w/ @weissweiler.bsky.social + @kmahowald.bsky.social

I'll be dreaming of them once Swedish winter starts...