If ChatGPT Atlas, or another AI browser, takes off we'll have websites that are built to be AI-friendly, or AI-first.

Imagine YouTube, Canva, Revolut, or Amazon but designed with both the human's and the AI's ease of use in mind.

What will the web look like?

Any iOS / XCode experts here that can help me figure out why I'm not seeing the "StoreKit Configuration" option when editing app scheme?

Screenshot 1: what I see
Screenshot 2 & 3: what RevenueCat and Apple docs claim I should be seeing

🙏

And that's it - that's how Continue Autocomplete is made!

Keep reading some Github links to our codebase where you can dig into this in more detail 🧑‍💻

Sometimes, the model might take too long to respond.

Based on our data, if the user doesn’t receive a completion within the first 300ms they’re unlikely to accept it.

That’s why, if the stream is taking too long, we display whatever we have generated so far and fetch the rest in the background.

We have a filter for each one of the major problems.

Some of them fix the issues, some of them have the authority to end the stream and return an empty response instead.

The final prompts might looks like this. They contain a header with snippets and a prefix and suffix separated by a FIM marker:

Continue allows you to plug in a variety of LLMs and we need to generate the right template for each one of them.

We have some custom, model-specific templates for the most popular models, and a generic fallback template for the remaining models.

At the moment, our algorithm is simple. We use any git and clipboard snippet we are able to collect and if there are still tokens left, we fill the prompt with import definitions from the AST.

Next, we collect all the uncommitted git changes - they provide important context on user intent.

For example, if a user just created a ‘UserSettingsForm’ React component and now they’re on ‘UserSettingsPage.tsx’, the model can guess the user most likely wants to place the form there.

For example, if the function is ‘createUser(user: Partial<User>)’, we need to know the type of the User object, or the model is going to hallucinate a plausible-looking type on its own.

🔶 Place the context inside a template that the LLM can understand

🔶 Send the template to the LLM

🔶 Process LLM’s response (pass it through filters, clean it up, etc)

🔶 Display the result (most often) or kill the completion (sometimes, when the completion is subpar)

🔷 Prefix - everything that comes before the FIM marker

🔷 Suffix - everything that comes after the FIM marker

🔷 Template - prefix, suffix, and other tokens put together, to be consumed by the LLM

- Make it smart - autocomplete should correctly anticipate your intent
- Make it fast - getting a completion should be almost instant
- Fail gracefully - sometimes the LLM fails to provide a good completion and we should detect that situation and hide the completion from the user

Ever wondered how AI autocomplete works? In this thread I’ll walk you through how we work with LLMs at @continuedev.bsky.social to decipher user intent and provide them with useful completions.

Continue is open source so I’ll post links to relevant code on Github at the end of this thread.

I'm in the "polish smile" rabbit hole

In Poland a smile is considered an unnatural facial expression and I think that's beautiful.