Ruizhe Li
@ruizheli.bsky.social
Assistant Professor at University of Aberdeen | Postdoc at UCL | PhD at University of Sheffield | mechanistic interpretability & multimodal LLMs | https://www.ruizhe.space
Time to go to Vienna again! I’ll present one mechinterp work on 28th 17:00-18:30 Hall X4 X5
We have another work for multilingual instruction-following benchmark on 28th 14:00 at 1.15-16. Very honored to be involved in this oral work! Feel free to reach out & chat for mechinterp!
We have another work for multilingual instruction-following benchmark on 28th 14:00 at 1.15-16. Very honored to be involved in this oral work! Feel free to reach out & chat for mechinterp!
July 24, 2025 at 8:06 AM
Time to go to Vienna again! I’ll present one mechinterp work on 28th 17:00-18:30 Hall X4 X5
We have another work for multilingual instruction-following benchmark on 28th 14:00 at 1.15-16. Very honored to be involved in this oral work! Feel free to reach out & chat for mechinterp!
We have another work for multilingual instruction-following benchmark on 28th 14:00 at 1.15-16. Very honored to be involved in this oral work! Feel free to reach out & chat for mechinterp!
In our case study for located MLP layers in Qwen2 models, we identify several correct decoded tokens are gradually transferred from their Chinese format to the English version, such as 一只(A), 拥有(has) and 翅膀(wings) in the figure.
June 3, 2025 at 5:25 PM
In our case study for located MLP layers in Qwen2 models, we identify several correct decoded tokens are gradually transferred from their Chinese format to the English version, such as 一只(A), 拥有(has) and 翅膀(wings) in the figure.
In addition, we move forward to locate relevant attention heads and MLP layers using JSD from mechinterp view. We found that JSD-based mechinterp can identify context attribution-related attention heads and MLPs, which are mainly distributed around intermediate or higher layers.
June 3, 2025 at 5:24 PM
In addition, we move forward to locate relevant attention heads and MLP layers using JSD from mechinterp view. We found that JSD-based mechinterp can identify context attribution-related attention heads and MLPs, which are mainly distributed around intermediate or higher layers.
We evaluate our ARC-JSD on TyDi QA, Hotpot QA and MuSiQue datasets using Qwen2-1.5B/7B-IT and Gemma2-2B/9B-IT, which can achieve higher attribution acc than baseline.
June 3, 2025 at 5:23 PM
We evaluate our ARC-JSD on TyDi QA, Hotpot QA and MuSiQue datasets using Qwen2-1.5B/7B-IT and Gemma2-2B/9B-IT, which can achieve higher attribution acc than baseline.
🤔Is it possible to accurately and effectively attribute RAG response to relevant context without finetuning or further training surrogate model?
💡We propose an inference-time method called ARC-JSD using JSD for RAG context attribution, which only needs O(sent_num + 1)🚀
💡We propose an inference-time method called ARC-JSD using JSD for RAG context attribution, which only needs O(sent_num + 1)🚀
June 3, 2025 at 5:18 PM
🤔Is it possible to accurately and effectively attribute RAG response to relevant context without finetuning or further training surrogate model?
💡We propose an inference-time method called ARC-JSD using JSD for RAG context attribution, which only needs O(sent_num + 1)🚀
💡We propose an inference-time method called ARC-JSD using JSD for RAG context attribution, which only needs O(sent_num + 1)🚀
