Crucially, InstABoost achieves this control without degrading text quality. While other latent steering methods can cause generation fluency to drop sharply as you increase their strength, InstABoost maintains coherence while steering towards the instruction.
(6/7)

Across 15 tasks, InstABoost either outperforms or matches the best steering method (prompt or latent-based). For tasks where prompt and latent-based steering perform equivalently, InstABoost can even combine the strengths of both and outperform both categories of methods.
(5/7)

InstABoost steers an LLM in attention space, bridging the performance gap between latent and prompt-based steering. InstABoost can be implemented in ~3 lines of code which simply increases attention weight to an in-context instruction.
(3/7)

Existing steering methods are either prompt or latent-based (modifying the hidden state), but which is better? We show the answer depends on the task. The steering task landscape includes those which are latent-optimal, instruction-optimal, and equivalent.
(2/7)

Excited to share our new paper: "Instruction Following by Boosting Attention of Large Language Models"!

We introduce Instruction Attention Boosting (InstABoost), a simple yet powerful method to steer LLM behavior by making them pay more attention to instructions.
(🧵1/7)

On the other hand, NeSy prompting provides two key benefits atop foundation models:

Reliability: A symbolic program enables accurate, stable, and trustworthy results.

Interpretability: Explicit symbols provide a clear, debuggable window into the model's "understanding."
(7/9)

3️⃣ The Program Pitfall: Training neural nets in conjunction with a fixed program leads to "hallucinated" symbols, reaching the correct answer for the wrong reasons, similar to reasoning shortcuts.
(6/9)

2️⃣ The Data Pitfall: Training on small, specialized datasets encourages overfitting.
(5/9)

1️⃣ The Compute Pitfall: Training specialized NeSy models has diminishing returns. As foundation models scale, the gap between NeSy training and NeSy prompting disappears, making dedicated training a costly detour.
(4/9)

🧠 Foundation models are reshaping reasoning. Do we still need specialized neuro-symbolic (NeSy) training, or can clever prompting now suffice?
Our new position paper argues the road to generalizable NeSy should be paved with foundation models.
🔗 arxiv.org/abs/2505.24874
(🧵1/9)