Sweta Karlekar
@swetakar.bsky.social
Machine learning PhD student @ Blei Lab in Columbia University
Working in mechanistic interpretability, nlp, causal inference, and probabilistic modeling!
Previously at Meta for ~3 years on the Bayesian Modeling & Generative AI teams.
🔗 www.sweta.dev
Working in mechanistic interpretability, nlp, causal inference, and probabilistic modeling!
Previously at Meta for ~3 years on the Bayesian Modeling & Generative AI teams.
🔗 www.sweta.dev
Sorry John, that isn’t my area of expertise!
November 25, 2024 at 12:44 AM
Sorry John, that isn’t my area of expertise!
This is very interesting! Do you have any intuition as to whether or not this phenomenon happens only with very simple “reasoning” steps? Does relying on retrieval increase as you progress from simple math to more advanced prompts like GSM8K or adversarially designed prompts (like adding noise)?
November 24, 2024 at 4:29 PM
This is very interesting! Do you have any intuition as to whether or not this phenomenon happens only with very simple “reasoning” steps? Does relying on retrieval increase as you progress from simple math to more advanced prompts like GSM8K or adversarially designed prompts (like adding noise)?
Learning doesn’t have to mean explicit weight changes; ICL can be viewed as temporary implicit finetuning (arxiv.org/abs/2212.10559) or like a “state” change to the model instead of a weight change, akin to how learning happens in fast RL vs slow RL (www.cell.com/trends/cogni...).
Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers
Large pretrained language models have shown surprising in-context learning (ICL) ability. With a few demonstration input-label pairs, they can predict the label for an unseen input without parameter u...
arxiv.org
November 22, 2024 at 10:31 PM
Learning doesn’t have to mean explicit weight changes; ICL can be viewed as temporary implicit finetuning (arxiv.org/abs/2212.10559) or like a “state” change to the model instead of a weight change, akin to how learning happens in fast RL vs slow RL (www.cell.com/trends/cogni...).
Hi! Our lab does Bayesian stuff :) Could you add Dave Blei's lab to this pack as well if it's not already full? @bleilab.bsky.social
November 20, 2024 at 3:38 PM
Hi! Our lab does Bayesian stuff :) Could you add Dave Blei's lab to this pack as well if it's not already full? @bleilab.bsky.social
Could you add Dave Blei's lab to this pack as well if it's not already full? @bleilab.bsky.social
November 20, 2024 at 3:37 PM
Could you add Dave Blei's lab to this pack as well if it's not already full? @bleilab.bsky.social
Could you add Dave Blei's lab to this pack as well if it's not already full? @bleilab.bsky.social
November 20, 2024 at 3:36 PM
Could you add Dave Blei's lab to this pack as well if it's not already full? @bleilab.bsky.social
Could you add Dave blei's lab to this pack as well if it's not already full! @bleilab.bsky.social
November 20, 2024 at 3:36 PM
Could you add Dave blei's lab to this pack as well if it's not already full! @bleilab.bsky.social
Oh, I’ve been meaning to check out that YouTube series—thanks! Also sadly, there's no class website, but I can share the "super quick intro to mech interp" presentation I made. It’s somewhat rough, but hopefully, it gets the main points across! sweta.dev/files/intro_...
sweta.dev
November 20, 2024 at 3:08 PM
Oh, I’ve been meaning to check out that YouTube series—thanks! Also sadly, there's no class website, but I can share the "super quick intro to mech interp" presentation I made. It’s somewhat rough, but hopefully, it gets the main points across! sweta.dev/files/intro_...
Reposted by Sweta Karlekar
📢 Post-Bayesian online seminar series coming!📢
To stay posted, sign up at
tinyurl.com/postBayes
We'll discuss cutting-edge methods for posteriors that no longer rely on Bayes Theorem.
(e.g., PAC-Bayes, generalised Bayes, Martingale posteriors, ...)
Pls circulate widely!
To stay posted, sign up at
tinyurl.com/postBayes
We'll discuss cutting-edge methods for posteriors that no longer rely on Bayes Theorem.
(e.g., PAC-Bayes, generalised Bayes, Martingale posteriors, ...)
Pls circulate widely!
Mailing list contact information
Information to be added to the post-Bayes mailing list.
tinyurl.com
November 19, 2024 at 8:22 PM
📢 Post-Bayesian online seminar series coming!📢
To stay posted, sign up at
tinyurl.com/postBayes
We'll discuss cutting-edge methods for posteriors that no longer rely on Bayes Theorem.
(e.g., PAC-Bayes, generalised Bayes, Martingale posteriors, ...)
Pls circulate widely!
To stay posted, sign up at
tinyurl.com/postBayes
We'll discuss cutting-edge methods for posteriors that no longer rely on Bayes Theorem.
(e.g., PAC-Bayes, generalised Bayes, Martingale posteriors, ...)
Pls circulate widely!
Adding Neel Nanda's favorite paper list as well: www.alignmentforum.org/posts/NfFST5...
(6/n)
(6/n)
An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2 — AI Alignment Forum
This post represents my personal hot takes, not the opinions of my team or employer. This is a massively updated version of a similar list I made two…
www.alignmentforum.org
November 20, 2024 at 2:55 PM
Adding Neel Nanda's favorite paper list as well: www.alignmentforum.org/posts/NfFST5...
(6/n)
(6/n)