Liam Bai
@liambai.bsky.social
I wrote a new blog post on using the logit lens to interpret protein language models!
liambai.com/logit-lens/
It has some interactive visualization you can play around with.
liambai.com/logit-lens/
It has some interactive visualization you can play around with.
May 22, 2025 at 8:08 PM
I wrote a new blog post on using the logit lens to interpret protein language models!
liambai.com/logit-lens/
It has some interactive visualization you can play around with.
liambai.com/logit-lens/
It has some interactive visualization you can play around with.
Reposted by Liam Bai
It's long seemed that molecular biology is a natural home for ML interpretability research, given the maturity of human-constructed models of biological mechanisms—permitting direct comparison with their ML-derived counterparts—unlike vision and NLP. Our first foray below👇.
Can we learn protein biology from a language model?
In new work led by @liambai.bsky.social and me, we explore how sparse autoencoders can help us understand biology—going from mechanistic interpretability to mechanistic biology.
In new work led by @liambai.bsky.social and me, we explore how sparse autoencoders can help us understand biology—going from mechanistic interpretability to mechanistic biology.
February 10, 2025 at 4:15 PM
It's long seemed that molecular biology is a natural home for ML interpretability research, given the maturity of human-constructed models of biological mechanisms—permitting direct comparison with their ML-derived counterparts—unlike vision and NLP. Our first foray below👇.
Reposted by Liam Bai
This might be the best paper on applying sparse autoencoders to protein language models. The authors identify how neural networks trained on amino acid sequences "discover" different features, some specific to individual protein families, other for substructures
www.biorxiv.org/content/10.1...
www.biorxiv.org/content/10.1...
February 10, 2025 at 12:15 PM
This might be the best paper on applying sparse autoencoders to protein language models. The authors identify how neural networks trained on amino acid sequences "discover" different features, some specific to individual protein families, other for substructures
www.biorxiv.org/content/10.1...
www.biorxiv.org/content/10.1...
Reposted by Liam Bai
Can we learn protein biology from a language model?
In new work led by @liambai.bsky.social and me, we explore how sparse autoencoders can help us understand biology—going from mechanistic interpretability to mechanistic biology.
In new work led by @liambai.bsky.social and me, we explore how sparse autoencoders can help us understand biology—going from mechanistic interpretability to mechanistic biology.
February 10, 2025 at 4:12 PM
Can we learn protein biology from a language model?
In new work led by @liambai.bsky.social and me, we explore how sparse autoencoders can help us understand biology—going from mechanistic interpretability to mechanistic biology.
In new work led by @liambai.bsky.social and me, we explore how sparse autoencoders can help us understand biology—going from mechanistic interpretability to mechanistic biology.