No, it was available to try out after the keynote by people that attended the event, some journalists briefly talked about it further, but didn’t share much concrete information about speed. We’ll have to wait until someone builds their own cluster & provides more detail.

I think framework had a demo running of full deepseek r1 671b running on a cluster of 4 of these.

This is a mobile CPU, meaning that it’s soldered, which is why you can’t really buy it separately, similar to how most other laptop cpu+mobos aren’t sold separately.

Framework does sell the mobo+cpu+ram separately though if you want to build your own cluster of them: frame.work/gb/en/produc...

Basically a “we’ve got nvidia project digits at home”

Indeed doesn’t make much sense as a regular desktop, but that AMD chip has a more powerful GPU attached than similar products. That makes it an interesting product for having lots of “vram” at a lower price. Token/s perf is probably a lot lower than your setup bc of lower vram bandwidth though.

Hmm, independent benchmarks seem to have the qwen32b distill about on par with 4o. Either way, seems like it could be good hardware for self hosted AI, but it’s still a ways away from larger sota models.

It’s a great model that outperforms o1-mini, including coding, and is far better than 4o. It’s not as good as o3-mini/full r1, but it’s great for a local model. The q4 will impact performance, but you should also be able to run the full r1-distill-qwen-32B, which is also still better than o1-mini.

Coming back to this, this might be a very budget friendly way to get 100gb+ of vram: frame.work/gb/en/deskto...

You can allocate up to 110gb of 128gb total to the GPU, making it possible to run models like deepseek-r1-distill-llama-70b at home with 4 bit quantitization.

Yes, it’s quite easy, see the linked Reddit post, they are running 3x p40s, which gives them 72gb of vram. If you get a good deal on the p40s, you can have 3 for three digits. This is a lot cheaper than you’d think.

People seem to be having a good time running a couple of these for locally running LLAMA models www.reddit.com/r/LocalLLaMA...

Using nvidia p40s is a lot cheaper than using 3090s, and they have just as much vram.

Available for $300-400 each, 24gb vram. www.ebay.com/p/12018725644

Does that also imply your heat pumps were continuously fighting against your AC then? I can’t imagine the temperature in your house being the same now as when your heat pumps were putting out 2-3x as much heat otherwise.