Allen Nie
banner
allenanie.bsky.social
Allen Nie
@allenanie.bsky.social
Stanford CS PhD working on RL and LLMs with Emma Brunskill and Chris Piech. Co-creator of Trace. Prev @GoogleDeepMind @MicrosoftResearch

Specifically
- Offline RL
- In-context RL
- Causality

https://anie.me/about
Unverified hot takes go to this account
I credit Omar @lateinteraction.bsky.social for this beautiful summary of the difference 🤣
December 11, 2024 at 2:14 AM
Hi Tim — Trace can optimize the control flow, whereas DSPy optimizes the modules in a fixed control flow (for now) 🙂 I would use DSPy for a supervised learning setup and Trace for an RL-like task (when there’s a clear definition of reward and feedback).
December 11, 2024 at 2:13 AM
Trace performs inference-time optimization — not directly updating weights of the underlying neural network. It updates the agentic workflow (python functions, prompts to LLMs and etc)
December 11, 2024 at 12:53 AM
We are happy to give a talk or have a 1:1 chat if you are interested in learning what Trace is and/or how to use it! Trace has already been presented at the UW Robotics Colloquium and ServiceNow. #foundermode for Open-Source Software! Time to build 🔧 and ship 🚀!
December 10, 2024 at 7:52 PM
This open-source project is a joint effort with
@chinganc_rl
and Adith, the MSR RL group. We are presenting Trace at the NeurIPS Expo Demo this afternoon 3pm-5pm PT. We have MUGs, T-SHIRTs, and STICKERs!

🌐 microsoft.github.io/Trace/
👨‍💻 github.com/microsoft/Tr...
GitHub - microsoft/Trace: End-to-end Generative Optimization for AI Agents
End-to-end Generative Optimization for AI Agents. Contribute to microsoft/Trace development by creating an account on GitHub.
github.com
December 10, 2024 at 7:52 PM
Once you build an agent with Trace, you can use ANY LLM optimizer you want. With the release of Trace 0.1.3, we introduce TextGrad (github.com/microsoft/Tr...) as an optimizer for the RL agent, along with OPRO and OptoPrime.
December 10, 2024 at 7:52 PM
What enables Trace to be an RL-style agentic library? We use **Generative Optimization** techniques (LLM as an optimizer) to derive an analog to RL's policy gradient algorithm. The agent makes a move, receives feedback/reward, and updates its parameters.
December 10, 2024 at 7:52 PM
In Trace, you define an Agent with declarative Python functions using Trace primitives. Trace provides flexible ways to mark what you want to change -- for example, we mark two prompts and two functions below as trainable.
December 10, 2024 at 7:52 PM
True RL agents learn online -- continuously changing themselves to improve upon the feedback (reward) from a user or an environment. Why haven't people done this in the LLM "Agentic" libraries? We wondered the same and developed Trace -- a true *RL-style* agentic framework.
December 10, 2024 at 7:52 PM
An honor to have you here!! Welcome 🙏🙏
November 30, 2024 at 4:35 AM
📌
November 25, 2024 at 2:38 PM
Can I get added? Not NLP but still working with LLMs on the RL side.
November 25, 2024 at 2:19 AM
📌
November 24, 2024 at 1:15 AM
Totally — it’s a great list 😊
November 23, 2024 at 6:24 PM
Hi, I’m one of the main maintainers of Trace: github.com/microsoft/Tr... and will use this platform to promote it and engage with the OSS community 🫡
GitHub - microsoft/Trace: End-to-end Generative Optimization for AI Agents
End-to-end Generative Optimization for AI Agents. Contribute to microsoft/Trace development by creating an account on GitHub.
github.com
November 23, 2024 at 2:19 PM