If you (or anyone you know) would like to do your PhD in a vibrant city with great weather, next to the sea, and the mountains, make sure to apply to UCLA and mark my name as a potential advisor :)

Stephen (@stephenmell.bsky.social) is also looking for postdoc opportunities so reach out to him if you are working in the intersection of PL, ML, and systems and are looking to hire! He is amazing!

You can find a lot more information on our work, proofs, details, additional evaluation, and more 😮 in our paper that Stephen (@stephenmell.bsky.social) presented at OOPSLA a few weeks ago. You can also read it here: dl.acm.org/doi/pdf/10.1...

We prove correctness by proving that the semantics is confluent 🚰 , meaning that even if different execution steps are happening with our opportunistic evaluation, they will all lead to the same final program state.

But why is that correct you may think to yourself? 🤔 Thanks for asking, I wouldn’t have thought to tell you!

We have proved that our approach is ✨ correct ✨, meaning that the behavior of the program does not change when executed with opportunistic evaluation.

We then use everyone’s favorite technique (Church ⛪ encodings) to desugar control flow of a high level language down to our calculus enabling streaming and parallelization for programs that involve LLMs and other ML models.

Our key insight is an evaluation strategy that is neither totally strict, nor totally lazy, and can automatically exploit parallelization and streaming. We build this evaluation strategy on top of a core calculus with first class support for external calls (like ones to an LLM).

To address this, we built a novel 🏇opportunistic 🏇 evaluation system that automatically parallelizes independent calls and streams their results! Our system reduces the execution time of this script from 51s to 9s, and the first output arrives on the terminal after only 1.3s!

In the example script in the image above, all calls to the LLM are completely 🐜🐜🐜 sequentialized 🐜🐜🐜 and there is no streaming. This script takes about 51 seconds to execute, and it took more than 7 seconds to start seeing output written in the terminal.

There is some alignment between these two threads indeed I think! Happy to chat whenever :)

In this paper we don't yet build such guardrails but primarily talk about what these guardrails should/could look like! We also focus on guardrails at the systems and PL level, instead of improving the agents. The paper is short so please read it and share your thoughts with us :)

😠😠😠😠

The data includes statistical information about the shapes and sizes of real-world Durable Functions applications. We hope that it will provide new insights to the community and indicate directions for future work!