Florian Eichin
@florian-eichin.com
PhD candidate at LMU Munich. Representations, model and data attribution, training dynamics.
Strong opinions on coffee and tea ☕
https://florian-eichin.com
Strong opinions on coffee and tea ☕
https://florian-eichin.com
is the bike doing fine though?? 😥
July 23, 2025 at 12:58 PM
is the bike doing fine though?? 😥
Reposted by Florian Eichin
📝Probing LLMs for Multilingual Discourse Generalization Through a Unified Label Set
🔎Do LLMs encode and generalize discourse knowledge across languages?
👥 @florian-eichin.com @janetlauyeung.bsky.social @mhedderich.bsky.social @barbaraplank.bsky.social
🔗 arxiv.org/abs/2503.10515
📁Main - Long
🔎Do LLMs encode and generalize discourse knowledge across languages?
👥 @florian-eichin.com @janetlauyeung.bsky.social @mhedderich.bsky.social @barbaraplank.bsky.social
🔗 arxiv.org/abs/2503.10515
📁Main - Long
July 23, 2025 at 12:30 PM
📝Probing LLMs for Multilingual Discourse Generalization Through a Unified Label Set
🔎Do LLMs encode and generalize discourse knowledge across languages?
👥 @florian-eichin.com @janetlauyeung.bsky.social @mhedderich.bsky.social @barbaraplank.bsky.social
🔗 arxiv.org/abs/2503.10515
📁Main - Long
🔎Do LLMs encode and generalize discourse knowledge across languages?
👥 @florian-eichin.com @janetlauyeung.bsky.social @mhedderich.bsky.social @barbaraplank.bsky.social
🔗 arxiv.org/abs/2503.10515
📁Main - Long
Reposted by Florian Eichin
📄 [ACL 2025 main] LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks (doi.org/10.48550/arX...)
LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks
There is an increasing trend towards evaluating NLP models with LLMs instead of human judgments, raising questions about the validity of these evaluations, as well as their reproducibility in the case...
doi.org
July 18, 2025 at 10:19 AM
📄 [ACL 2025 main] LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks (doi.org/10.48550/arX...)
Reposted by Florian Eichin
📄 [ACL 2025 main] Circuit compositions: Exploring Modular Structures in Transformer-Based Language Models (doi.org/10.48550/arX...)
Circuit Compositions: Exploring Modular Structures in Transformer-Based Language Models
A fundamental question in interpretability research is to what extent neural networks, particularly language models, implement reusable functions through subnetworks that can be composed to perform mo...
doi.org
July 18, 2025 at 10:19 AM
📄 [ACL 2025 main] Circuit compositions: Exploring Modular Structures in Transformer-Based Language Models (doi.org/10.48550/arX...)
Reposted by Florian Eichin
Here's the preprint---basically, you can just use concept erasure (github.com/EleutherAI/c...) to remove lang id
arxiv.org/abs/2507.01234
arxiv.org/abs/2507.01234
The Medium Is Not the Message: Deconfounding Text Embeddings via Linear Concept Erasure
Embedding-based similarity metrics between text sequences can be influenced not just by the content dimensions we most care about, but can also be biased by spurious attributes like the text's source ...
arxiv.org
July 3, 2025 at 9:10 AM
My MSc-thesis has been turned into a paper (whose framing you will probably not enjoy) that introduces a method which can be viewed as an unsupervised solution to a similar problem. Will share later to avoid biasing review process
July 3, 2025 at 2:59 PM
My MSc-thesis has been turned into a paper (whose framing you will probably not enjoy) that introduces a method which can be viewed as an unsupervised solution to a similar problem. Will share later to avoid biasing review process
Interesting! And indeed very relevant as it enables control over the similarity modeled by the embeddings. Figure 2 is really cool. Which base embeddings were used for this?
July 3, 2025 at 2:57 PM
Interesting! And indeed very relevant as it enables control over the similarity modeled by the embeddings. Figure 2 is really cool. Which base embeddings were used for this?
Haha can't wait. Let's continue the discussion at ACL!
July 2, 2025 at 9:30 AM
Haha can't wait. Let's continue the discussion at ACL!
Also, I remember your other ACL 2025 paper which shows that the LLM approach comes with problems for topic quality, too? Very interesting read arxiv.org/abs/2502.14748
Large Language Models Struggle to Describe the Haystack without Human Help: Human-in-the-loop Evaluation of Topic Models
A common use of NLP is to facilitate the understanding of large document collections, with a shift from using traditional topic models to Large Language Models. Yet the effectiveness of using LLM for ...
arxiv.org
July 2, 2025 at 8:35 AM
Also, I remember your other ACL 2025 paper which shows that the LLM approach comes with problems for topic quality, too? Very interesting read arxiv.org/abs/2502.14748
Yeah, agreed and aware of your work :) though as established above, emb+clustering has its niche in large scale analysis with factors like multilinguality. There, LDA tends to have problems and TopicGPT is too expensive.
July 2, 2025 at 8:32 AM
Yeah, agreed and aware of your work :) though as established above, emb+clustering has its niche in large scale analysis with factors like multilinguality. There, LDA tends to have problems and TopicGPT is too expensive.
Awesome! And yes, I totally understand and agree with the scepticism towards that
July 2, 2025 at 8:12 AM
Awesome! And yes, I totally understand and agree with the scepticism towards that