Randall Hunt
banner
randall.dev
Randall Hunt
@randall.dev
CTO at Caylent. We're hiring! Formerly @pytorch, @awscloud, @mongodb, @SpaceX.
If not friend, why friend shape?

Also no.
February 1, 2025 at 4:28 AM
Have a great holiday
December 24, 2024 at 10:54 PM
Why should we *not* build new data centers?
December 24, 2024 at 10:51 PM
The idea that these data centers are metabolizing the planet is hyperbolic. Agree or disagree?
December 24, 2024 at 10:50 PM
Are you intentionally misconstruing my point to create a strawman?
December 24, 2024 at 10:46 PM
> "What is happening all around us is the creation of a technical infrastructure that is directly competing with humans for basic resources like water, energy, and land-pushing the limits of a planet already under strain."

I disagree with this quote re: energy and land. Water maybe.
December 24, 2024 at 10:39 PM
The original supposition is that AI workloads are metabolizing the planet. I disagree that AI workloads are a significant driving factor of energy consumption globally. No one is claiming GPU hungry workloads are less energy intensive. The discussion is what portion of consumption does it represent?
December 24, 2024 at 10:22 PM
When faced with evidence and hypothesis, if your general activity is to go ad hominem and attack the integrity of the person you disagree with, then you kill any form of discussion.

I'm open to being wrong, are you also open to being wrong? Do you hold your opinions or do they hold you?
December 24, 2024 at 10:14 PM
I will be direct.

When the evidence doesn't support your priors, maybe you should reexamine your assumptions?

Across hyperscalers and smaller operators there is no one number that is correct. The error bars are massive. Anyone claiming to have a perfect idea of the numbers is likely incorrect.
December 24, 2024 at 10:12 PM
Yes. I worked:
1) at AWS
2) on the Pleiades supercomputer
3) on infra for PyTorch and model training at Meta across on prem and AWS
4) on prem DCs for SpaceX
5) currently for several customers who run large ML workloads across AWS and on prem GPU farms
December 24, 2024 at 10:08 PM
I am an expert in this field and I am explaining to you that it is difficult to quantify due to a lack of public data.

One can hypothesize though and the fact is that power hungry GPU based workloads are not as widely deployed as overall data center capacity. At least not yet.
December 24, 2024 at 10:03 PM
You're right I shouldn't have used an overloaded (pun intended) term there.
December 24, 2024 at 8:03 PM
Workload here refers to load in the electrical sense
December 24, 2024 at 8:01 PM
The term workload here is not workload in a devops sense but workload in an electrical sense
December 24, 2024 at 8:00 PM
I'm not saying it's zero or meaningless to think about these things... but it's not metabolizing the world. That's a statement that sounds good but has little to no backing in actual numbers. Would 2% of global power usage be the first target for efficiency gains?
December 24, 2024 at 7:56 PM
Here's my hypothesis:

Total large GPU deployments in data centers
-----
Total # of data centers

My bet is the denominator is *much* bigger than the numerator. There are fixed power costs for each of the DCs in the denominator that don't go down.
December 24, 2024 at 7:54 PM
I don't? What evidence would you use to support that conclusion?

1. Building data centers takes time.
2. Not all data centers have massive GPU deployments, most around the world basically function as CDNs.
December 24, 2024 at 7:48 PM