Lucas Weber
@lucweber.bsky.social
Post-doc working at Fraunhofer-IIS on LLM posttraining
Reposted by Lucas Weber
Here's our work accepted to #ICLR2025!
We look at how intrinsic dimension evolves over LLM layers, spotting a universal high-dimensional phase.
This ID peak is where:
- linguistic features are built
- different LLMs are most similar,
with implications for task transfer
🧵 1/6
We look at how intrinsic dimension evolves over LLM layers, spotting a universal high-dimensional phase.
This ID peak is where:
- linguistic features are built
- different LLMs are most similar,
with implications for task transfer
🧵 1/6
February 2, 2025 at 6:53 PM
Here's our work accepted to #ICLR2025!
We look at how intrinsic dimension evolves over LLM layers, spotting a universal high-dimensional phase.
This ID peak is where:
- linguistic features are built
- different LLMs are most similar,
with implications for task transfer
🧵 1/6
We look at how intrinsic dimension evolves over LLM layers, spotting a universal high-dimensional phase.
This ID peak is where:
- linguistic features are built
- different LLMs are most similar,
with implications for task transfer
🧵 1/6