For a short moment, ML had a future at bluesky, but the allure of the fascist-owned community square is just too strong. I am disappointed at the weakness on display...

This is quite a neat demonstration of the counter-intuitive dangers of using current ML tech in critical situations.

You'd think that more reflective = more visible, but visibility is a problem for humans. For ML more reflective mean more unusual, which mean the system doesn't know what to do […]

Earlier post, but in recent talks I'm encountering more and more organizations that are losing their last technical people. You can outsource a lot, but most places have a core thing that they should really own. And once your own technical department is no longer viable, you are hosed. The […]

A very interesting read for anyone into the corporate political history of modern AI. Quite clearly shows what was up at OpenAI back in the day:
https://www.lesswrong.com/posts/5jjk4CDnj9tA7ugxr/openai-email-archives-from-musk-v-altman

A few notes of interest:
* Greg Brockman and Ilya […]

On my way back, listened to the Gwern podcast (nothing groundbreaking overall), and a thesis I had heard before did strike me more ferociously this time: if LLM scaling holds for a few more years, LLMs might be able to generate Gwern-level essays on their own.

Since Gwern is a very impressive […]

Spent a few days in an actually big city, and was surprised to see OsmAnd app buckling under the density of the information. Every swipe on my pocket computer a painful exercise in patience. Fortunately, OrganicMaps worked just fine and snappy. Why was I using OsmAnd anyway...

After a long day spent amidst concrete blocks and diesel particulate emissions, the airy ambience and the high notes of Minuk's Voz de las Montañas hit different.

A small respite: https://www.youtube.com/watch?v=2Cz3PC_cxB8

Here's everything I've figured out about the new OpenAI o1 family of models so far https://simonwillison.net/2024/Sep/12/openai-o1/

Mistral launched Pixtral 12B today, their first multi-modal (text plus images) vision LLM. It’s Apache 2 licensed and a 25GB download. I put together some notes on the release here: https://simonwillison.net/2024/Sep/11/pixtral/

Was curious about how candle performs vs onnxruntime (via ort) on video, so have put together a small experimental repo using #gstreamer for video decode/encode and candle/ort for inference: https://github.com/slckl/gstreamed_rust_inference

Ort is roughly 3-4x as fast using cuda, and 5-6x as […]

Contemplating slowly bangerposting every then and now.

Tonight we dine on jazz gone through the ringer, by Grischa Lichtenberger: https://www.youtube.com/watch?v=_bAQ7OwixxM
and other organic substances:
https://www.youtube.com/watch?v=ibInHhy5Jmw

Grischa reminds many of Autechre, both are […]

@tedunderwood.me

Paper does touch on this in the outer (aka parallel/unconscious processing) vs inner (serial/conscious) brain, and the question how can the latter be implemented using the same substrate (massively parallel neurons) as the former. Clearly the total rate of information […]

@tedunderwood.me very fun paper, thank you!

Maybe I'm naive, but it feels like total rate of cognition should include the supporting machinery enabling the 10 bits of behavior.

Without the predictive/context machinery of humans (aka "unconscious processing" in the paper), the 10 bits can't be […]

A fun article exploring the puzzle that a) neurons are fast b) our senses can transmit perhaps a gigabit/sec in parallel, but c) the overall throughput of human behavior is ~10 bits/second. Some bottleneck in our central nervous system is forcing us to process serially. arxiv.org/abs/2408.10234 🧪 +

It's very easy for AI companies to solve issues around IP law in ways that ensure corporate enclosure of the intellectual commons. If we want the commons *not* to be enclosed, we need to strengthen fair use instead.

Excitement started building yesterday around rumors that Llama 3.1 (open source) would outperform proprietary models like GPT-4o. Now we have the public release, so people can judge for themselves. 1/2 #machinlearning #MLSky

Meta announcement: ai.meta.com/blog/meta-ll...

Two new Mistral models today: Codestral Mamba is an Apache 2 licensed 7B code model using a Mamba architecture (not transformers!), and MathΣtral is for "math reasoning and scientific discovery"

My notes here, including an update to my llm-mistral plugin […]