Ribhu
@ribhulahiri.com
Improving decision making in medicine @ Miimansa | ex-Founder@kaksha.ai | 🎓UCSD, PlakshaTLF
While these are just 3 thoughts I had while reading the above blog, what do y'all think about the "simple parts connected by clean interfaces" bit in the context of LLM-based systems today? What are some patterns and anti-patterns you have noticed?
October 18, 2025 at 12:17 PM
While these are just 3 thoughts I had while reading the above blog, what do y'all think about the "simple parts connected by clean interfaces" bit in the context of LLM-based systems today? What are some patterns and anti-patterns you have noticed?
Unlike deterministic APIs, LLMs can return valid JSON that's semantically wrong. Strict schemas catch this at the interface, not in your business logic.
October 18, 2025 at 12:17 PM
Unlike deterministic APIs, LLMs can return valid JSON that's semantically wrong. Strict schemas catch this at the interface, not in your business logic.
We NEED this in the modern AI stack.
October 18, 2025 at 12:17 PM
We NEED this in the modern AI stack.
More specifically the structure of it. What I mean is that when you start architecting an app, one of the first tasks we do is to create API contracts. Creation of a schema to communicate between components is what allows things to independently grow without any fear of breakdown of operability.
October 18, 2025 at 12:17 PM
More specifically the structure of it. What I mean is that when you start architecting an app, one of the first tasks we do is to create API contracts. Creation of a schema to communicate between components is what allows things to independently grow without any fear of breakdown of operability.
Coming back to the thread, what makes any full stack application work well, even with hundreds of components, modules, microservices, and so on, is the reliability of the information.
October 18, 2025 at 12:17 PM
Coming back to the thread, what makes any full stack application work well, even with hundreds of components, modules, microservices, and so on, is the reliability of the information.
They operate at the logit layer of any open-weights model to ensure that the specified schema is "almost" deterministically followed.
October 18, 2025 at 12:17 PM
They operate at the logit layer of any open-weights model to ensure that the specified schema is "almost" deterministically followed.
3️⃣ Lack of reliability in outputs. Which is kind of the point Remy is making because that's what @dottxtai.bsky.social does so well. I would urge folks to try out the outlines library if you haven't already.
October 18, 2025 at 12:17 PM
3️⃣ Lack of reliability in outputs. Which is kind of the point Remy is making because that's what @dottxtai.bsky.social does so well. I would urge folks to try out the outlines library if you haven't already.
Adding simpler flows from the beginning saves us this back-and-forth of adding validation checks and output parsers
October 18, 2025 at 12:17 PM
Adding simpler flows from the beginning saves us this back-and-forth of adding validation checks and output parsers
A single complex prompt that needs 3 retries costs more than 3 simple prompts that work first time.
October 18, 2025 at 12:17 PM
A single complex prompt that needs 3 retries costs more than 3 simple prompts that work first time.
Unfortunately, what follows later on is having to add flows to retry, or post-process the output to refine it in a way where we get the output in the desired form and fidelity. Which leads us to more prompts anyway.
October 18, 2025 at 12:17 PM
Unfortunately, what follows later on is having to add flows to retry, or post-process the output to refine it in a way where we get the output in the desired form and fidelity. Which leads us to more prompts anyway.
2️⃣ More prompts = More tokens = More cost. This is the mind-killer. This mental model is what leads engineers and products people to fit everything into less prompts.
October 18, 2025 at 12:17 PM
2️⃣ More prompts = More tokens = More cost. This is the mind-killer. This mental model is what leads engineers and products people to fit everything into less prompts.
These are your initial set of sub-tasks, which you can later refine as needed.
October 18, 2025 at 12:17 PM
These are your initial set of sub-tasks, which you can later refine as needed.
Now, you go about your task the way you – the master – would, and note down all the steps you needed to get it done. The first thought would be to combine a few of them. DON'T
October 18, 2025 at 12:17 PM
Now, you go about your task the way you – the master – would, and note down all the steps you needed to get it done. The first thought would be to combine a few of them. DON'T
A mental model that I find helpful with this is the master-apprentice model. Here, you are the master and the LLM (you pretending to be one) is the apprentice.
October 18, 2025 at 12:17 PM
A mental model that I find helpful with this is the master-apprentice model. Here, you are the master and the LLM (you pretending to be one) is the apprentice.
The ONLY way to get better at this is to train this muscle of breaking down tasks into the absolute singular task that is simple and stateless
October 18, 2025 at 12:17 PM
The ONLY way to get better at this is to train this muscle of breaking down tasks into the absolute singular task that is simple and stateless
And this trickles down to designing prompts as well.
October 18, 2025 at 12:17 PM
And this trickles down to designing prompts as well.
The reason why engineering and product managers exist. When given a goal, a lot of us sub-optimally break it down, based on our cognitive affordances.
October 18, 2025 at 12:17 PM
The reason why engineering and product managers exist. When given a goal, a lot of us sub-optimally break it down, based on our cognitive affordances.
1️⃣ Breaking down a task into simpler sub-tasks. The hardest of the 3.
October 18, 2025 at 12:17 PM
1️⃣ Breaking down a task into simpler sub-tasks. The hardest of the 3.
And this simplification of work is by no means easy. There are 3 main factors that make it harder:
October 18, 2025 at 12:17 PM
And this simplification of work is by no means easy. There are 3 main factors that make it harder:
Agentic patterns that can alleviate this, can sometimes be worse, where an agent can have instructions on pursuing multiple things at the same time.
October 18, 2025 at 12:17 PM
Agentic patterns that can alleviate this, can sometimes be worse, where an agent can have instructions on pursuing multiple things at the same time.
There is almost a sort of pride that prompting folks take in being able to do everything in "one-shot" and not having to rely on multiple turns.
October 18, 2025 at 12:17 PM
There is almost a sort of pride that prompting folks take in being able to do everything in "one-shot" and not having to rely on multiple turns.
Software engineering was built on this principle, but it's something I don't often see AI engineers follow (myself included). Many times there is a tendency to add every single instruction and decision point in a single prompt.
October 18, 2025 at 12:17 PM
Software engineering was built on this principle, but it's something I don't often see AI engineers follow (myself included). Many times there is a tendency to add every single instruction and decision point in a single prompt.
I was prompted to structure these thoughts after reading this blog by @remilouf.bsky.social:
blog.dottxt.ai/do-one-thing...
blog.dottxt.ai/do-one-thing...
Do One Thing Well
blog.dottxt.ai
October 18, 2025 at 12:17 PM
I was prompted to structure these thoughts after reading this blog by @remilouf.bsky.social:
blog.dottxt.ai/do-one-thing...
blog.dottxt.ai/do-one-thing...