Tianyi Zhou
@zhoutianyi.bsky.social
Assistant Professor of Computer Science at the University of Maryland and UMIACS; Research in AI, Machine Learning, NLP, Multi-modality; Previous: Research Scientist at Google; PhD from the University of Washington
🚨We will present "Your MoE LLM Is Secretly an Embedding Model for Free" at #ICLR2025 Oral (1.77%) this April in Singapore:
1⃣Routing weights (RW) in MoE provide training-free embedding complementary to widely-used hidden states (HS)
2⃣MoEE (RW + HS) beats standalone HS by +23%
1⃣Routing weights (RW) in MoE provide training-free embedding complementary to widely-used hidden states (HS)
2⃣MoEE (RW + HS) beats standalone HS by +23%
February 13, 2025 at 3:26 PM
🚨We will present "Your MoE LLM Is Secretly an Embedding Model for Free" at #ICLR2025 Oral (1.77%) this April in Singapore:
1⃣Routing weights (RW) in MoE provide training-free embedding complementary to widely-used hidden states (HS)
2⃣MoEE (RW + HS) beats standalone HS by +23%
1⃣Routing weights (RW) in MoE provide training-free embedding complementary to widely-used hidden states (HS)
2⃣MoEE (RW + HS) beats standalone HS by +23%