The Sound Defense
LightNews LightNews
thesounddefense.bsky.social
The Sound Defense
@thesounddefense.bsky.social
580 followers 1.1K following 760 posts
Thoughts about gaming, politics, comedy, cooking and other such. Also I named Sharknado 2 and I'll never do anything else as important in my life.
Posts Replies Media Videos
thesounddefense.bsky.social
NO

SLEEP

'TIL PERSONAL REVELATION
December 1, 2025 at 5:34 PM
Reposted by The Sound Defense
peiferlabunc.bsky.social
A wonderful tribute
From Wiki In October 2010, Munroe's fiancée was diagnosed with stage three breast cancer; there had been no prior family history.[51][52] The emotional effect of her illness was referenced in the comic panel "Emotion", published 18 months later in April 2012.[53] In September 2011, he announced that they had married.[54] In November 2012, Munroe published a comic entitled "Two Years", reflecting on their relationship since his wife's breast cancer diagnosis.[55] He revisited the subject in December 2017 with a comic entitled "Seven Years"[56], in November 2020 in a comic entitled "Ten Years"[57], and again in November 2025 in a comic entitled "Fifteen Years".[58]
November 26, 2025 at 11:02 PM
Reposted by The Sound Defense
edzitron.com
Yet what worries me the most is that NVIDIA claimed to have "shipped 6 million Blackwell GPUs" in the last year. That's 6GW to 12GW of IT load capacity - the data centers do not exist, there is not enough space, there is not enough power. Where are the GPUs?
www.wheresyoured.at/the-haters-g...
NVIDIA Claims To Have SHIPPED 6 Million Blackwell GPUs In The Last 4 Quarters - Which Would Amount To 6GW to 12GW Of Data Center Capacity, With 1GW Taking 2.5 Years and $40bn-$50bn - Meaning Millions Of Blackwell GPUs Are Sat Waiting To Be Installed Nevertheless, putting all that aside, NVIDIA CEO Jensen Huang claims that NVIDIA has shipped 6 million Blackwell GPUs — and according to CNBC, that specifically refers to AI GPUs. Shipped. They have left NVIDIA’s warehouses. These chips are in flight. They are real. Where the fuck are they? So, Stargate Abilene is meant to have 1.2GW of power, and each building is 440,000 square feet according to developer Lancium, and it appears based on some reporting that each building will be 100MW of IT load, though I’m having trouble getting a consistent answer here. In any case, we can do some napkin maths! 100MW = 50,000 Blackwell GPUs (I’m going to guess B200s), making 6 million Blackwell GPUs somewhere in the region of 12GW of IT load, and because data centers need 30% or more power than their IT loads (to cover for that “design day” i mentioned earlier), that means 15.6GW of power is required to make the last four quarters of NVIDIA GPUs sold turn on. Now, let’s rejig that maths and assume I’m somehow wrong by some level, and each data center at Abilene is 75MW of IT Load. Even then, that drops us to 9GW of IT load, and 11.7GW of power required. Then, perhaps, there are millions of B100 GPUs — less powerful ones, and therefore, less power-hungry — involved. So we could foreseeably say that 6GW is the number. I think that number is wrong, by the way, I’m just being generous. In any case, in an indepth dig through Data Center Map and masses of Googling, I have been unable to establish how much actual, real AI data center capacity actually exists, but I can for sure tell you that there are not even a million Blackwell GPUs currently in operation.
November 24, 2025 at 5:04 PM
Reposted by The Sound Defense
edzitron.com
GPUs are expensive to install, expensive to run...and the AI services that run on them *don't make much money from consumers* and lose incredible amounts of money due to the inconsistent and hard-to-predict costs of LLMs. There's no fixing this problem.
www.wheresyoured.at/the-haters-g...
AI Software Does Not Make Enough Money To Cover The Cost of Renting GPUs To Run AI Services, And Costs Vary Dramatically Buddy, I hear you yelling that I should sing Enter Sandman, and here we go! I’ve already talked about this so many times, but there really is just not that much money in AI services, in part because the services aren’t worth paying more for, and in part because costs are difficult to control. Large language models are a uniquely tough economic situation for startups — or any business, for that matter. As a business, you do not “pay to access an LLM” as a one-off cost, but pay on a per-million-tokens basis, either for input tokens (what a user loads into the model, such as text or files) or output tokens (what the model outputs through a process called “inference,” which is when it creates said output based on what a user asks). For example, OpenAI charges $1.25 per million input and $10 per million output tokens for GPT-5.1. A good rule of thumb is that one token is about ¾ of a word, and it’s important to remember that generating code is also generating characters (so letters or numbers), and one token equals one character. I also should note that the amounts that OpenAI charges for its models are not necessarily indicative of the costs of running them. In other kinds of software, you could charge a user a monthly subscription fee — $20-a-month, for example — and know roughly what a user might cost, because the unit costs are consistent. The cost of Netflix showing a 4K movie to a person in Idaho, or Meta loading Instagram for somebody in Sweden, for example. While there is some regional variation, it’s not particularly massive (at least, compared to the inconsistencies of genAI), and it’s something you can anticipate ahead of time. The “cost of generating an answer from a Large Language Model” is variable every single time based on multiple criteria: The user’s query, which might be simple, such as “what if Scooby Doo lay eggs,” or as complex as “generate the code for a piece of software that will help me keep track of The Infidel Marmaduke The Dog, Inshallah.” The model’s interpretation of the user’s input, which is when hallucinations become a problem. In the Scooby Doo egg example, a model might decide it needs to use a “web search tool” to find sources that might explain whether a dog can lay an egg, or whether Scooby Doo has, in previous canon, laid eggs. I sound like I’m joking around, but this example would lead the model on a wild goose chase, with each “tool access” and piece of information generated requiring you to generate more output tokens. In the Marmaduke Software example, remember that code is extremely token-intensive, and this is not a particularly detailed prompt. It does not specify what kind of software would track Marmaduke, and has numerous additions (infidel, Inshallah) that the Large Language Model might choose to interpret and build something that the user doesn’t want. As a result, the user might receive software that doesn’t do what the user wants, causing the user to go back and demand more software. The user’s response to the output it receives. If you’re asking a model a question, you’re likely to ask a follow-up question. And based on what the model says, you may have more follow-up questions. Every follow-up question is an opportunity for the user to demand more output from the model, and for the model to decide, based on maths, to generate more answers. If a model doesn’t do what a user wants, the user is also likely to keep asking the model stuff until it either gets it right or the user gets annoyed. This isn’t necessarily the LLM’s fault. Like any bad customer, the user hasn’t been clear about what exactly the software should do, but might be mad tha… Services try to ease this burden by imposing limits (like ChatGPT has), such as the amount of requests a user can make within a five hour period, the amount of information that can make up an input from a user, the amount of information from the conversation that a model can consider at once (called a context window), and what information can be loaded in (text, images, PDFs, and so on). The problem is that even with those limits, every single AI service is effectively intricately prompting the model on the back-end to deliver the service itself using something called a “service prompt” — think of it as a kind of primer on what kind of service you want it to perform, and also what limitations you have on said service. There is no way, it appears, to make a model be more cost-conscious, other than to tell it to not do something or give shorter answers. While you can tell a coding model to “write efficient code,” and even define specifically what efficiency is and looks like, said model is still, every single time probabilistically working out A) what the user wants and B) what the service wants the model to do as a result. In practice, this means that there’s very few ways to prevent a user from costing you far more than they’re paying you each month without directly charging them for the cost of using the models, which is far more than users are used to paying for. This means that most AI software companies are charging a monthly fee with amounts of “requests” or “credits” that a user can make. This pricing structure obfuscates the true cost of the models, all while allowing users to do whatever they want within those boundaries. For example, AI coding platform Augment Code had one user paying $250 a month that cost it $15,000 a month, and even with further changes to the product (switching from letting users make “requests” to giving them “credits), the company does not appear to be profitable. You may think that these companies will simply “raise their prices…
November 24, 2025 at 5:04 PM
Reposted by The Sound Defense
edzitron.com
A dirty little secret of the AI boom is that it appears everybody loses money on buying AI GPUs, as the incredible upfront costs (billions) put every data center in a deep, dark hole - and because GPUs are so power-hungry, it's difficult to make any margin at all.
www.wheresyoured.at/the-haters-g...
Why It’s So Unprofitable To Run AI Services Using NVIDIA’s (or anyone else’s) GPUs Before we go any further, it’s important to remember that the rest of the cloud computing industry is relatively high-margin, and that’s because the stuff it sells is, by and large, nowhere near as expensive as running GPUs. That’s because you really don’t need that much compute to serve a normal website, or a normal web application. Cloud providers can take a resource — like physical memory, or storage, or a single virtual CPU core — and sell it at a pretty steep mark-up relative to the cost of actually providing it. Ordinary computing resources are cheaper, and they can be divided across more users. As a result, the margins are better and the upfront costs are lower. By contrast, a single AI application — like one that generates a video, or an image, or that processes a large codebase — may use multiple GPUs simultaneously. The whole division of resources that makes cloud computing so efficient (and so lucrative) simply doesn’t apply here. And that’s without mentioning the cost of ownership, or indeed, the additional power costs that come from GPUs, or the cost of keeping them cool. I could keep on going. By virtue of their scale, AI data centers often require additional infrastructure that doesn’t really apply with conventional computing gear. This may involve a greater number of transformers and substations to deal with the insane power demands (all of which are extremely expensive, and have painfully long lead times), or on-site power generation infrastructure. The Stargate Abilene facility, as well as xAI’s Colossus facility in Memphis, Tennessee, all have (or will have) on-site natural gas turbines, as well as pipelines that provide the gas they burn, and all of this costs money. Again, these turbines have an incredible lead time — which is why we’re seeing creative alternatives spring up, like former jet engines being repurposed into stationary generators. Sidenote: I have no idea how efficient those converted engines actually are, but the Interesting Engineering article linked above notes that the engines being used by one such remanufacturer are the General Electric CF6-80C2 — an 80s vintage design that flew with the McDonnell Douglas MD-11 (which first flew in 1988) and the Boeing 747-400 (an aircraft that debuted the same year, and that has largely left passenger service because it’s not a particularly fuel-efficient plane to run). Obviously, there are variables that affect fuel efficiency in aviation that don’t apply to a stationary, ground-based power generator. Still, I’m curious about the economics of this, and whether these repurposed turbines have similar fuel consumption habits to, say, a purpose-built natural gas turbine! As I explained before the premium break, one does not simply “plug in” a GPU, nor does one just buy “a” GPU. You usually buy thousands, tens of thousands, or hundreds of thousands of these things, and the more you buy, the more likely it is you have to build a new data center to house them. While previous generations — Ampere (A100) and Hopper (H100, H200) — could be integrated into older data centers, new generations like Blackwell (2024) and Vera Rubin (coming next year) are so power-hungry and generate so much heat that they require specialized liquid cooling systems (which means using sealed tubes of specific coolant liquid to cool something), which in turn requires bigger, more power-intensive data center architectures to handle them. Oh, and the chips are tens of thousands of dollars more expensive than the previous generation, while also having higher electricity and cooling costs. Important detail to add: remember all that stuff I said about parallel processing and doing a bunch of maths at once? Doing all that maths is very intensive on these GPUs, meaning they’re running at full tilt a lot of the time. That’s a lot of heat. I get this all sounds super complex and confusi… Because you can’t reuse the old servers and cooling, you have to both buy a bunch of new stuff and build a bunch of new stuff. You can’t just use the old cooling either, so you need a bigger data center, meaning more construction (unless you’re just building a new one). Construction is expensive and takes time, meaning that while you may have bought all those new GPUs, you have to wait to get them all installed. When your construction is done, you now need more power than before. Once your data center is built, you have spent an incredibly large amount of money and taken a whole bunch of time. When you turn on the GPU, it immediately starts costing you money. This means that every GPU purchase starts in a deep, dark hole. While a GB200 rack containing 72 GPUs might cost $3 million, its costs balloon even further when you factor in all the stuff you had to build around it so that it A) turns on, B) functions properly and C) doesn’t overheat and turn off. It’s difficult to say how much more because things like power and construction vary from place to place, and something as simple as, say, permitting can have an outsized effect on the price of construction, but my napkin maths says there are tens of thousands of dollars of costs at minimum for each GB200 rack. So, yeah, when you finally turn a GPU on, you’re tens of thousands of dollars in the hole already. And then we get to the muddy part — how much do these fucking things cost to run? The answer is “nobody seems to want to tell us,” and also “it’s complicated.” It’s hard to tell exactly how much one GPU costs, despite many companies (like Microsoft, Google or Amazon, or AI-specialist companies like CoreWeave) selling hourly access to GPUs, because data centers are full of them, and because a GPU isn’t just a GPU, it’s also connected to a CPU, a server, storage, and many other pieces that need to exist for it to do anything.
November 24, 2025 at 5:04 PM