Michael Castelle
@mcastelle.bsky.social
it won't unless you really want to describe the linear/nonlinear vector trajectories of high-dimensional (and often multimodal!) input tokens as "symbolic"
September 30, 2025 at 1:59 PM
it won't unless you really want to describe the linear/nonlinear vector trajectories of high-dimensional (and often multimodal!) input tokens as "symbolic"
when they say Stewart Brand prefigured the Internet they definitely got that part right
September 25, 2025 at 5:14 PM
when they say Stewart Brand prefigured the Internet they definitely got that part right
The history of computer science is full of “theorems” that, once sufficiently popular in a given subfield, appear to obstruct conceptual progress for about a decade
September 25, 2025 at 8:31 AM
The history of computer science is full of “theorems” that, once sufficiently popular in a given subfield, appear to obstruct conceptual progress for about a decade
Actually a great book FYI: www.amazon.com/Compass-Impr... . Much like Walter Pitts, Elaine May benefited from UofC's laissez-faire approach to student enrollment
The Compass: The Improvisational Theatre that Revolutionized American Comedy (Centennial Publications of the University of Chicago Press)
The Compass: The Improvisational Theatre that Revolutionized American Comedy (Centennial Publications of the University of Chicago Press) [Coleman, Janet] on Amazon.com. *FREE* shipping on qualifying offers. The Compass: The Improvisational Theatre that Revolutionized American Comedy (Centennial Publications of the University of Chicago Press)
www.amazon.com
September 21, 2025 at 6:45 PM
Actually a great book FYI: www.amazon.com/Compass-Impr... . Much like Walter Pitts, Elaine May benefited from UofC's laissez-faire approach to student enrollment
Very back-of-the-envelope math but going by recent official stats, Chicago appears to have over 10x crashes-per-registered-vehicle than London
September 13, 2025 at 4:29 PM
Very back-of-the-envelope math but going by recent official stats, Chicago appears to have over 10x crashes-per-registered-vehicle than London
We’ve also been training on a mix of code and natural language (cleaned/augmented or not) since GPT-3 which should have been extremely weird to everyone. But I agree that humanists are well placed to contrast the long history of “reasoning” with whatever it is that currently goes under that name
September 5, 2025 at 3:35 AM
We’ve also been training on a mix of code and natural language (cleaned/augmented or not) since GPT-3 which should have been extremely weird to everyone. But I agree that humanists are well placed to contrast the long history of “reasoning” with whatever it is that currently goes under that name
I was disappointed less in the humanists themselves than in the computers they had to use
September 5, 2025 at 3:08 AM
I was disappointed less in the humanists themselves than in the computers they had to use
My position (as a disappointed bystander to much of the initial project) is that it is only now that “Digital Humanities” can truly begin
September 5, 2025 at 2:59 AM
My position (as a disappointed bystander to much of the initial project) is that it is only now that “Digital Humanities” can truly begin
I guess I’m personally on your side here but for whatever reason, people who love counting seem to be slightly uncomfortable with having counting taken away from them (such as, for example, when you do gradient descent on hundreds of random token sequences simultaneously and repeat ad infinitum)
September 5, 2025 at 2:45 AM
I guess I’m personally on your side here but for whatever reason, people who love counting seem to be slightly uncomfortable with having counting taken away from them (such as, for example, when you do gradient descent on hundreds of random token sequences simultaneously and repeat ad infinitum)
Yes, and I believe that sense-making machines made from layers of linear and non-linear rotations/rectifications of high-dimensional vectors are quite intellectually (and semiotically) inimical to the many cultural descendants of symbol-centric logic/probability/computation/tabulation
September 5, 2025 at 2:36 AM
Yes, and I believe that sense-making machines made from layers of linear and non-linear rotations/rectifications of high-dimensional vectors are quite intellectually (and semiotically) inimical to the many cultural descendants of symbol-centric logic/probability/computation/tabulation
If what you mean by “statistical” is “learns from data somehow”, sure. Otherwise, disagree. Calling softmax() occasionally is pretty distant from actual probability theory; and there’s nothing continuous about 1990s “statistical NLP” vs. today’s NLP.
September 5, 2025 at 1:52 AM
If what you mean by “statistical” is “learns from data somehow”, sure. Otherwise, disagree. Calling softmax() occasionally is pretty distant from actual probability theory; and there’s nothing continuous about 1990s “statistical NLP” vs. today’s NLP.
It's worth questioning how much of the Sutton & Barto-style reinforcement learning framework actually remains in the present LLM fine-tuning context IMO (not saying I have the answer to this question... but my intuition is that it's less than one might think!)
September 4, 2025 at 4:30 PM
It's worth questioning how much of the Sutton & Barto-style reinforcement learning framework actually remains in the present LLM fine-tuning context IMO (not saying I have the answer to this question... but my intuition is that it's less than one might think!)
A lot of existing "digital humanities" is also (overtly or covertly) indebted to the symbolic/tabular/statistical forms of computing that contemporary large-scale connectionist AI deeply problematizes and/or doesn't much care about.
September 4, 2025 at 4:27 PM
A lot of existing "digital humanities" is also (overtly or covertly) indebted to the symbolic/tabular/statistical forms of computing that contemporary large-scale connectionist AI deeply problematizes and/or doesn't much care about.
Another (moderately more scholarly) classic is Chang (2008) "The myth of the boiling point" www.jstor.org/stable/43423...
The myth of the boiling point on JSTOR
HASOK CHANG, The myth of the boiling point, Science Progress (1933-), Vol. 91, No. 3 (2008), pp. 219-240
www.jstor.org
August 26, 2025 at 5:36 PM
Another (moderately more scholarly) classic is Chang (2008) "The myth of the boiling point" www.jstor.org/stable/43423...
Don't worry, such content has been reclassified at the system prompt level as purely metaphorical x.com/AmandaAskell...
August 6, 2025 at 6:57 PM
Don't worry, such content has been reclassified at the system prompt level as purely metaphorical x.com/AmandaAskell...
They are, but they don't seem to actually give access to historians of computing so (outside of Cortada's books) they constitute more of a scholarly black hole than not
July 24, 2025 at 9:15 AM
They are, but they don't seem to actually give access to historians of computing so (outside of Cortada's books) they constitute more of a scholarly black hole than not
Just checked and my council has actually been installing a couple hundred new hangars throughout 2024-2025, including an upcoming one around the corner 😎
July 17, 2025 at 5:15 AM
Just checked and my council has actually been installing a couple hundred new hangars throughout 2024-2025, including an upcoming one around the corner 😎
FWIW it’s a nice idea but in practice most of these have very long waiting lists, I got an email 5 years after applying for a spot on my block. (Possibly fixable with more enforcement of semi-regular use, idk)
July 16, 2025 at 2:52 PM
FWIW it’s a nice idea but in practice most of these have very long waiting lists, I got an email 5 years after applying for a spot on my block. (Possibly fixable with more enforcement of semi-regular use, idk)
(P.S. one of the greatest outsider sociology books of all time, saying www.amazon.com/Playing-at-W... )
June 27, 2025 at 2:42 PM
(P.S. one of the greatest outsider sociology books of all time, saying www.amazon.com/Playing-at-W... )
doesn't all that stuff (including the Northern IL/Southern WI wargaming variants in the US) derive from Kriegspiel / von Clausewitz? Already Lutheran.
June 27, 2025 at 2:39 PM
doesn't all that stuff (including the Northern IL/Southern WI wargaming variants in the US) derive from Kriegspiel / von Clausewitz? Already Lutheran.
Not sure Princeton Univ Press will accept my candidate title "Trust in 14,000-Dimensional Vectors"
June 27, 2025 at 8:28 AM
Not sure Princeton Univ Press will accept my candidate title "Trust in 14,000-Dimensional Vectors"
My assumption is that this kind of extraction is only possible when model developers don't make a serious/extensive attempt to thoroughly deduplicate the pretraining data (though you can still expect it for e.g. passages in novels that are widely quoted elsewhere).
June 26, 2025 at 7:58 PM
My assumption is that this kind of extraction is only possible when model developers don't make a serious/extensive attempt to thoroughly deduplicate the pretraining data (though you can still expect it for e.g. passages in novels that are widely quoted elsewhere).