Nicholas Lourie
@nicholaslourie.bsky.social
Better empirical methods for deep learning & NLP. PhD at NYU. Advised by He He and @kyunghyuncho.bsky.social. Prev: @ai2.bsky.social.
I build things. 🤖
I build things. 🤖
Pinned
📄🔈✨ Deep learning is an empirical science, but we rely on basic empirical methods. What might a better foundation—a simple theory—for empirical work look like?
@kyunghyuncho.bsky.social, He He, and I move towards one in "Hyperparameter Loss Surfaces Are Simple Near their Optima" at #COLM2025!
🧵1/9
@kyunghyuncho.bsky.social, He He, and I move towards one in "Hyperparameter Loss Surfaces Are Simple Near their Optima" at #COLM2025!
🧵1/9
📄🔈✨ Deep learning is an empirical science, but we rely on basic empirical methods. What might a better foundation—a simple theory—for empirical work look like?
@kyunghyuncho.bsky.social, He He, and I move towards one in "Hyperparameter Loss Surfaces Are Simple Near their Optima" at #COLM2025!
🧵1/9
@kyunghyuncho.bsky.social, He He, and I move towards one in "Hyperparameter Loss Surfaces Are Simple Near their Optima" at #COLM2025!
🧵1/9
October 8, 2025 at 8:14 PM
📄🔈✨ Deep learning is an empirical science, but we rely on basic empirical methods. What might a better foundation—a simple theory—for empirical work look like?
@kyunghyuncho.bsky.social, He He, and I move towards one in "Hyperparameter Loss Surfaces Are Simple Near their Optima" at #COLM2025!
🧵1/9
@kyunghyuncho.bsky.social, He He, and I move towards one in "Hyperparameter Loss Surfaces Are Simple Near their Optima" at #COLM2025!
🧵1/9
Reposted by Nicholas Lourie
Can we predict emergent capabilities in GPT-N+1🌌 using only GPT-N model checkpoints, which have random performance on the task?
We propose a method for doing exactly this in our paper “Predicting Emergent Capabilities by Finetuning”🧵
We propose a method for doing exactly this in our paper “Predicting Emergent Capabilities by Finetuning”🧵
November 26, 2024 at 10:37 PM
Can we predict emergent capabilities in GPT-N+1🌌 using only GPT-N model checkpoints, which have random performance on the task?
We propose a method for doing exactly this in our paper “Predicting Emergent Capabilities by Finetuning”🧵
We propose a method for doing exactly this in our paper “Predicting Emergent Capabilities by Finetuning”🧵