I talk about this a lot, but that is why I think Apple has the right approach with Foundation Models. There is a lot of potential for built in on device models to perform specific tasks. These models will be more useful in the longrun compared to expensive data centers.

Yep, and you can even send private cloud computer images, and it will describe the content of those images. Mona for Mastodon uses a cool trick that will send an image out of their app to a shortcut they install, get the description back, and bring that text back into their app.

I agree, and what is really truly amazing is that this is version 1. I wonder what we will see in the 27 releases of Foundation Models.

Tool calling wouldn't be the most useful in games, but custom system prompts and fine tuned adapters would be very useful.

I've reviewed a lot of the materials online and did not see it, but it can be done since we use it in Perspective Intelligence. It definitely isn't designed for it, but that is why it is fun to see what's possible.

Also, examples that are given is to use FM as a game NPC, so in a way, that can be viewed as a sort of chat situation as well.

I think Apple had specific use cases in mind for Foundation Models, but I have not found in their documentation where they specifically say to not use it as a chatbot. We had to get a bit creative to get it to work as one, but it is really amazing to see what it is capable of.

I've worked with the model a lot of the last few months, and it definitely has limits, but it is kind of amazing for being a model that is built in to new iPhones. I wish Apple would manage sessions for us though.

There are some great and amazing resources for Swift here on Bluesky, and I'm going to do a full SwiftUI course later this year. I did one last year and just moved it over to Google Classroom if you want to go through it. You can find it at classroom.google.com/c/NzQ1NzA4Nz...