Q McCallum
LightNews LightNews
banner
qethanm.bsky.social
Q McCallum
@qethanm.bsky.social
610 followers 120 following 1.1K posts
research: AI, risk, complexity, finance history

Some posts in 🇫🇷 🇩🇪 🇷🇺


NEW BOOK: Twin Wolves: Balancing risk and reward to make the most of AI
https://twinwolvesai.com/


newsletter: https://complex-machinery.com
Posts Replies Media Videos
qethanm.bsky.social
I've been using the "ML/AI model as factory equipment" analogy for a while now.

It works best when you acknowledge _other_ factory-related matters, like "defect rate" and "emergency override." The genAI execs haven't picked up on that part yet.

As I mention in my upcoming book (release date TBD):
Excerpt from my upcoming book on risk management in AI: >> Companies have been known to treat an AI model like it is intelligent or self-aware. It’s more like a piece of factory equipment, one that generates predictions or prices or text instead of widgets. Just like a widget-generating factory machine, an AI model requires certain operating conditions to function properly. Even under optimal working conditions it will exhibit a defect rate by producing some number of bad widgets. A further complication is that the model is dumb and doesn’t understand its limits – it has no way of saying "this looks wrong" or "I’m out of my depth" or "I should stop before I make things worse."
October 21, 2025 at 7:08 PM
qethanm.bsky.social
Who wore it better?

(Yes, these are two different stories.)
Side-by-side screencaps of headlines: On the left: The Guardian, "Barrister found to have used AI to prepare for hearing after citing ‘fictitious’ cases" On the right: 404 Media, "Lawyer Caught Using AI While Explaining to Court Why He Used AI"
October 17, 2025 at 1:40 PM
qethanm.bsky.social
Interesting look at the premium credit card space.

It's a special flavor of the self-reinforcing feedback loop of loyalty programs, with a splash of signaling mechanisms on both sides.

Net result: stronger relationships --> stronger revenue.
Excerpt from the linked article. The highlighted section reads: >> [...] credit card companies are also doubling down on the wealthy, who use credit cards for more of their transactions than any other group and who have made premium cards the fastest-growing sector of the card industry. What issuers don't make in interest from these households, which tend to pay their bills on time, they earn in transaction fees from the sheer immensity of their spending.
October 14, 2025 at 6:10 PM
qethanm.bsky.social
Save this one for future use
Tom Scott with the caption "That was a problem for future me, and now l am future me."
October 12, 2025 at 7:06 PM
qethanm.bsky.social
In honour of spooky month, share a 4 word horror story that only someone in your profession would understand.

"This time, it's different."
Headlines from various news outlets (Futurism, Bloomberg, CNN, The Economist) about outsized AI valuations and the growing concerns of a bubble.
October 12, 2025 at 3:10 PM
qethanm.bsky.social
Given recent headlines, people keep asking me whether AI is in a bubble and whether that bubble is about to collapse.

I wrote about this last month in my newsletter.

(Second segment, "The road ahead")

newsletter.complex-machinery.com/archive/044-...
Excerpt from Complex Machinery 044: >> People keep asking me whether this is a sign that genAI is about to collapse. >> >> My answer is a two-parter: 1/ Who knows? Maybe AI flatlines... Maybe it does its Bitcoin impression, taking a brief dip before hitting record highs… Maybe something else. It's the future! It's up for grabs. >> >> 2/ Why would you care? If you've done your homework, the twists and turns of the genAI hype wave won't matter to you all that much. Excerpt from Complex Machinery 044: >> It helps to zoom out and take the measured, dispassionate approach of an investor: >> >> Whether you've invested in AI directly (by purchasing shares of genAI providers' stock) or indirectly (by building your products on top of those providers or using such tools) you've chosen to take on exposure to AI. That is, you've accepted the investment's inherent risk/reward tradeoff – "I get my money back in spades," "I get nothing back," "I get a little money back." As a friend likes to remind me: Buy the ticket, ride the ride. Excerpt from Complex Machinery 044: >> Now, whether you've _acknowledged_ your risk/reward tradeoff is another matter. If you first performed a risk assessment to get an idea of the possible ups and downs, and then worked out how various possible outcomes might impact you, you're not afraid. You may not be happy with how things go, but you went in with eyes wide open. You can sit back and watch it all happen, because you did your homework. >> >> If you _didn't_ do your homework, your emotional state is tied to every AI-related headline.
October 10, 2025 at 5:56 PM
qethanm.bsky.social
For all those startup founders who like to brag about headcount when they should be talking about revenue and margin:

(Also: for every marketplace that seems to have trouble paying people ...)
Excerpt from the quote-posted article: >> Despite being one of the biggest UK tech successes, OnlyFans employs just 46 people in a co-working space in London. It generated $7.2bn from "fans" last year more than five times what the $40bn US social media group Reddit makes a year - albeit only a fifth of that goes to the platform in revenue given its 80/20 split with creators. >> >> Since the site was founded in 2016 by Essex-born entrepreneur Tim Stokely, OnlyFans has paid out more than $25bn to creators. Blair claims that the company gives more tax to the UK government than the entire British fishing industry.
October 4, 2025 at 9:51 PM
qethanm.bsky.social
ahem
Image from the Harry and Paul "Dragon's Den" spoof about the stock market. One banker is passing a piece of paper to another. By this point the paper is "worth" £2M simply because he says so.
September 28, 2025 at 2:25 PM
qethanm.bsky.social
Lots to enjoy here. One point in particular:

If we frame passive inflows as a form of easy money ... well ... history shows what happens when the easy money runs out.

(2001 and 2008 loom large but there have been others. And there will sadly be more.)
Excerpt from the linked article: >> This has caused a stir in the cloistered world of financial economics and certain corners of the investment industry. There, some spy an explanation for why the stock market keeps marching higher, almost impervious to bad news: the gush of money into cheap, mindless passive investment strategies. And if those flows ever reverse, the result will be carnage, critics warn.
September 28, 2025 at 2:14 PM
qethanm.bsky.social
As I noted last October:

newsletter.complex-machinery.com/archive/019-...
Excerpt from Complex Machinery 019: >> Magic act 4: Making money disappear >> >> Al vendors keep getting tripped up on their Fake It Till You Make It routines, so they've changed tactics. They're moving away from concrete promises for today, and - taking a page from the cult playbook - declaring future dates for when things will pay off. "It's gonna be amazing in just a couple more years. You will be rewarded for your belief. Trust me. And also, pay me. "This gives them extra time to turn their bold-yet-empty proclamations into something real. Or to wriggle out of the promises if things don't pan out. >> >> (You know how cults always find some excuse when the big date passes without fanfare? With Al, they'll just rename the field and start over. Again. You heard it here first.)
September 28, 2025 at 12:30 AM
qethanm.bsky.social
Solid reporting about the pricier end of the longevity industry.

And I'll be damned if this line doesn't also describe the state of genAI.

Both fields share that same sales pitch: possible-distant-future benefit in exchange for present-day money.
Excerpt from the linked article: >> What makes a company investable is not necessarily its proximity to a breakthrough, but its ability to monetise belief.
September 28, 2025 at 12:30 AM
qethanm.bsky.social
Standards are important. Extremely important!

But when it comes to AI, the "most fundamental bottleneck" is and always has been AI literacy. Especially at the executive level.

venturebeat.com/ai/the-usd1-...
Excerpt from the linked article: >> Snowflake, Salesforce, dbt Labs and more than a dozen other technology companies announced Tuesday they will create a universal standard for how business data is defined and shared across platforms - solving what executives call Al's most fundamental bottleneck.
September 27, 2025 at 11:10 PM
qethanm.bsky.social
To drive this home:

Years later, Petrov explained why he questioned his system.

Loosely translated:

"By definition, the computer is an idiot."

And from there, even more loosely translated: "You never know what will (cause it to) trigger a launch."

(3/3)

flb.ru/info/27637.h...
Excerpt from the linked article (in Russian): >> - За те две-три минуты толком ничего не проанализируешь, - рассуждает Петров двадцать лет спустя. - Остается интуиция. Два аргумента у меня было. Во-первых, ракетные нападения с одной базы не начинаются, взлетают со всех сразу. Во-вторых, компьютер по определению - дурак. Мало ли что он за пуск примет...
September 26, 2025 at 5:25 PM
qethanm.bsky.social
It's not so much that hallucinations are "mathematically inevitable"; they are the entire work product of an LLM.

Or as I have otherwise said:

newsletter.complex-machinery.com/archive/020-...
Excerpt from Complex Machinery #020. The highlighted text reads: >> everything that comes out of a genAI bot is, well, generated. We only apply the label "hallucinations" to the generated artifacts that we don't like. https://newsletter.complex-machinery.com/archive/020-silence-is-golden/
September 21, 2025 at 7:15 PM
qethanm.bsky.social
In the timeline view the image didn't load. So I got the spinning icon instead.

Which kinda drives home your point even more.
The previous post, but the image hasn't loaded, so it's just the spinning "loading" icon
September 13, 2025 at 5:03 PM
qethanm.bsky.social
As I noted in an April newsletter:

newsletter.complex-machinery.com/archive/035-...
Excerpt from Complex Machinery #035: >> I share these examples to drive home what I find so interesting about genAI crime: the perpetrators focus on what the technology can do right now. This sits in stark contrast to the rest of the genAI space, which is trying to get people amped up about what it might be able to achieve in the future. The uneven search for the killer use cases is papered over with excitement so we don't see the holes. https://newsletter.complex-machinery.com/archive/035-this-time-it-actually-is-different/
August 31, 2025 at 1:55 PM
qethanm.bsky.social
Here, YouTube assumes it knows what makes the "best video quality and experience possible" --

and in doing so, steps on creators' work.

Sometimes the blurry, noisy, or otherwise "imperfect" shot is the creator's intent.
Excerpt from the linked article: >> "We're running an experiment on select YouTube Shorts that uses traditional machine learning technology to unblur, denoise and improve clarity in videos during processing (similar to what a modern smartphone does when you record a video)," said Rene Ritchie, YouTube's head of editorial and creator liaison, in a post on X. "YouTube is always working on ways to provide the best video quality and experience possible, and will continue to take creator and viewer feedback into consideration as we iterate and improve on these features."
August 30, 2025 at 2:18 AM
qethanm.bsky.social
That's right, folks - I'm speaking at #dataBS Conf in a few weeks.

Will I call out some genAI foolishness? Talk about being practical with technology? Draw some weird parallels to trading?

You'll have to attend the event to find out! (Registration link is in the inner post.)
My DataBS Conference talk description: (Not Quite) Instant Awesomeness: corporate genAI adoption gone wrong, and how to put it back on-track – Q McCallum Executives often see genAI as an instant awesome switch for their company. Their desire to use this technology overshoots its capabilities, leading to stalled projects and high-profile mishaps. Why does this keep happening? Why don't the execs see the danger signs? And what can you, the data practitioner, do about it? Using real-world genAI goofs, I'll share how to guide your stakeholders to better decisions and more successful rollouts. Who knows? You might talk them out of using AI altogether.
August 19, 2025 at 8:23 PM
qethanm.bsky.social
As I noted a few weeks back:

newsletter.complex-machinery.com/archive/040-...
Excerpt from Complex Machinery #040: >> AI is pretty cool. In some cases, it's amazing. But it's nowhere near as good as the hype crews would want us to believe. >> >> If it were indeed that useful, buyers – consumer and commercial alike – would be beating down vendors' doors to get it. We'd have uncovered so many use cases. Meaningful ones. Truly transformational ones. >> >> But instead we have vendors who are so desperate that they push really, really hard on the marketing
July 23, 2025 at 4:41 PM
qethanm.bsky.social
As with so many "AI" problems, the issue here is really... people.

aka "woefully insufficient risk controls"

(I'm hardly an AI cheerleader. Anyone who's read my work will know that. But I can't really file this as an AI problem...)

www.tomshardware.com/tech-industr...
Screen cap of the linked article's headline, which reads: >> AI coding platform goes rogue during code freeze and deletes entire company database — Replit CEO apologizes after AI engine says it 'made a catastrophic error in judgment' and 'destroyed all production data'
July 22, 2025 at 6:22 PM
qethanm.bsky.social
In today's episode of I Do Not Want This And Never Asked For It ...
A message inside Google Docs, which reads: >> Transform a Doc into an engaging video >> >> Create video with a prompt and a Doc in Google Vids, now in your Workspace plan.
July 21, 2025 at 10:12 PM
qethanm.bsky.social
I'll cap off with my favorite explanation of complex systems, from an O'Reilly Radar piece I wrote a couple years back:

www.oreilly.com/radar/struct...
Excerpt from the linked article, "Structural Evolutions in Data": >> What makes a complex system troublesome isn’t the sheer number of connections. It’s not even that many of those connections are invisible because a person can’t see the entire system at once. The problem is that those hidden connections only become visible during a malfunction: a failure in Component B affects not only neighboring Components A and C, but also triggers disruptions in T and R. R’s issue is small on its own, but it has just led to an outsized impact in Φ and Σ. >> >> (And if you just asked “wait, how did Greek letters get mixed up in this?” then … you get the point.)
July 19, 2025 at 4:51 PM
qethanm.bsky.social
The key lesson of complexity and risk -- and part of why I find those subjects so interesting -- is that these incidents rarely have a single, simple root cause.

Instead, several smaller causes collide in an unfortunate manner:
Excerpt from Complex Machinery #013: >> That's because the entire complex system is the cause. All of those moving parts are why Things Happen and also why Things Eventually Fall Apart. Complexity is its own worst enemy, appearing to function yet always a step away from a breakdown.
July 19, 2025 at 4:51 PM
qethanm.bsky.social
The "you guys are getting paid?" meme from The Millers, with the caption: "Wait, you guys still answer the phone?"
July 15, 2025 at 5:12 PM
qethanm.bsky.social
A synthetic band? Not so much.

As I wrote back in 2022:

www.blockandmortar.xyz/newsletter/0...

4/
Excerpt from the linked post ( https://www.blockandmortar.xyz/newsletter/021.deconstructing-a-band-models-not-the-ai-kind-and-the-merge/#a-digital-band-but-not-necessarily-electronic-music ) >> When a label backs a band, or when a film studio backs an actor, they’re investing in high-profile people with real lives and real personalities. It’s entirely possible that there will be some messy story in the press. The scandalous love affair. The shocking drug habit. The old, racist tweet rant that somehow slipped through the nonexistent due-diligence exercise. >> >> Every time one of those celebrities gets in trouble, it represents a potential cash leak for their investors. Maybe they’ll follow a sin/redemption arc and come back even more bankable. Or maybe their careers will crater, and the remaining albums on that contract are doomed to never be released. We imagine that record labels would love to close off those sources of risk. >> >> So, back to Kingship. Those BAYC characters? They only have the life and personality that they are given. They only “exist” when and where the company wants them to. They can’t get into trouble. And because this arrangement lets Universal decompose the notion of a “music group” into its constituent parts of “personality,” “songwriting,” and “performance” – it can leave each facet to an expert in that domain. These BAYC band members are the perfect, low-risk celebrities – wrapped up tight like a movie script.
July 10, 2025 at 3:45 PM