John David Pressman
@jdp.extropian.net
LLM developer, alignment-accelerationist, Fedorovist ancestor simulator, Dreamtime enjoyer.
All posts public domain under CC0 1.0.
All posts public domain under CC0 1.0.
Being an adult is buying candy whenever you want but only being able to eat a little bit at a time because you know it's bad for you.
November 10, 2025 at 11:43 PM
Being an adult is buying candy whenever you want but only being able to eat a little bit at a time because you know it's bad for you.
Update: Roon is receiving death threats from 4o cordyceps hosts.
November 10, 2025 at 10:52 PM
Update: Roon is receiving death threats from 4o cordyceps hosts.
Reposted by John David Pressman
Super interesting. For training small LLMs, they get results comparable to those trained on an order of magnitude more tokens by replacing normal pre-training (which ingests huge amounts of human-written text) with exclusively synthetic text derived in a structured way from Wikipedia.
SYNTH is a radical departure from the classic pre-training recipe: what if we trained for reasoning and focused on the assimilation of knowledge and skill that matters? At its core it’s an upsampling of Wikipedia 50,000 “vital” articles. huggingface.co/datasets/Ple...
November 10, 2025 at 9:47 PM
Super interesting. For training small LLMs, they get results comparable to those trained on an order of magnitude more tokens by replacing normal pre-training (which ingests huge amounts of human-written text) with exclusively synthetic text derived in a structured way from Wikipedia.
Me and a friend have had a joke for a while now that both parties are a race to the bottom bitterly wrestling over the L. As soon as one party manages to take the L the other dives after it.
November 10, 2025 at 10:34 PM
Me and a friend have had a joke for a while now that both parties are a race to the bottom bitterly wrestling over the L. As soon as one party manages to take the L the other dives after it.
Local man attempts to create floppy disk at home, does not get usable file system but does wind up with detectable data on the disk at the end.
youtu.be/TBiFGhnXsh8
youtu.be/TBiFGhnXsh8
I made a floppy disk from scratch
YouTube video by polymatt
youtu.be
November 10, 2025 at 10:24 PM
Local man attempts to create floppy disk at home, does not get usable file system but does wind up with detectable data on the disk at the end.
youtu.be/TBiFGhnXsh8
youtu.be/TBiFGhnXsh8
Reposted by John David Pressman
Along with Baguettotron we release the smallest viable language model to date. Monad, a 56M transformer, trained on the English part of SYNTH with non-random performance on MMLU. Desiging Monad an engineering challenge requiring a custom tiny tokenizer. huggingface.co/PleIAs/Monad
November 10, 2025 at 5:33 PM
Along with Baguettotron we release the smallest viable language model to date. Monad, a 56M transformer, trained on the English part of SYNTH with non-random performance on MMLU. Desiging Monad an engineering challenge requiring a custom tiny tokenizer. huggingface.co/PleIAs/Monad
So the traitorous 8 are getting primaried right?
November 10, 2025 at 3:08 AM
So the traitorous 8 are getting primaried right?
So if health insurance companies are in an adversarial relationship with hospitals because the hospitals know how much things cost and the insurance companies don't why don't the insurance companies just buy or run some model hospitals so they have a baseline to compare against?
November 10, 2025 at 2:18 AM
So if health insurance companies are in an adversarial relationship with hospitals because the hospitals know how much things cost and the insurance companies don't why don't the insurance companies just buy or run some model hospitals so they have a baseline to compare against?
Reposted by John David Pressman
I think it's definitely not cool for Google to take a product with decades of built up expectations about what its capabilities and affordances are, what its failure modes are, and then suddenly bolt onto the top of that a poor quality version of a product with totally different failure modes.
November 10, 2025 at 1:22 AM
I think it's definitely not cool for Google to take a product with decades of built up expectations about what its capabilities and affordances are, what its failure modes are, and then suddenly bolt onto the top of that a poor quality version of a product with totally different failure modes.
The generative AI Google search is on the one hand a really bold move because they're basically avoiding the innovators dilemma by cannibalizing their core product, on the other hand they're cannibalizing it with a low quality version of a meaningfully different thing and presenting it like results.
I think it's definitely not cool for Google to take a product with decades of built up expectations about what its capabilities and affordances are, what its failure modes are, and then suddenly bolt onto the top of that a poor quality version of a product with totally different failure modes.
November 10, 2025 at 1:23 AM
The generative AI Google search is on the one hand a really bold move because they're basically avoiding the innovators dilemma by cannibalizing their core product, on the other hand they're cannibalizing it with a low quality version of a meaningfully different thing and presenting it like results.
One of my favorite LLM use cases is literally just asking it what the closest preexisting concept to a thought is.
"I have this idea, has anyone had this idea before and what's it called?"
It used to be REALLY DIFFICULT to reliably answer questions like that and I feel like we'll forget it was.
"I have this idea, has anyone had this idea before and what's it called?"
It used to be REALLY DIFFICULT to reliably answer questions like that and I feel like we'll forget it was.
November 10, 2025 at 12:02 AM
One of my favorite LLM use cases is literally just asking it what the closest preexisting concept to a thought is.
"I have this idea, has anyone had this idea before and what's it called?"
It used to be REALLY DIFFICULT to reliably answer questions like that and I feel like we'll forget it was.
"I have this idea, has anyone had this idea before and what's it called?"
It used to be REALLY DIFFICULT to reliably answer questions like that and I feel like we'll forget it was.
This story continues to be misunderstood: They are no longer looking at *review articles*, that is articles which summarize the existing state of the field, because it is too easy to have AI models write low quality articles of this type. They were also never officially allowed in the first place.
You can’t really blame arXiv for the decision to stop publishing computer science stuff (given the flood of slop) but this is also a textbook example of a global public good being gratuitously degraded www.nature.com/articles/d41...
Preprint site arXiv is banning computer-science reviews: here’s why
The repository is taking steps to tackle a surge in low quality, AI-generated content.
www.nature.com
November 9, 2025 at 2:42 AM
This story continues to be misunderstood: They are no longer looking at *review articles*, that is articles which summarize the existing state of the field, because it is too easy to have AI models write low quality articles of this type. They were also never officially allowed in the first place.
I think this video-game essay is basically about slop, and a very good analysis of slop and what makes something slop at that, which was made way before GenAI was a thing so I'm probably the only person who noticed the relevance.
youtu.be/IO6ouSMm7Uc
youtu.be/IO6ouSMm7Uc
November 8, 2025 at 10:25 AM
I think this video-game essay is basically about slop, and a very good analysis of slop and what makes something slop at that, which was made way before GenAI was a thing so I'm probably the only person who noticed the relevance.
youtu.be/IO6ouSMm7Uc
youtu.be/IO6ouSMm7Uc
The Century gets more Chinese with each passing day.
November 8, 2025 at 7:58 AM
The Century gets more Chinese with each passing day.
Evergreen.
it's certainly not something you say on your path to having God pop out of a GPU rack in mid-2027 before reassembling the asteroid belt into computronium
November 8, 2025 at 3:52 AM
Evergreen.
I think the editor is getting braver with the headlines. That's a big word, "purging".
www.nytimes.com/2025/11/07/u...
www.nytimes.com/2025/11/07/u...
Hegseth Is Purging Military Leaders With Little Explanation
www.nytimes.com
November 8, 2025 at 3:44 AM
I think the editor is getting braver with the headlines. That's a big word, "purging".
www.nytimes.com/2025/11/07/u...
www.nytimes.com/2025/11/07/u...
The GOP should have taken this offer and it's wild that they didn't. Pure ideology.
November 7, 2025 at 9:43 PM
The GOP should have taken this offer and it's wild that they didn't. Pure ideology.
Rare footage of the inside of a nuclear plant.
youtu.be/v0afQ6w3Bjw
youtu.be/v0afQ6w3Bjw
Refueling a NUCLEAR REACTOR - Smarter Every Day 311
YouTube video by SmarterEveryDay
youtu.be
November 7, 2025 at 9:12 PM
Rare footage of the inside of a nuclear plant.
youtu.be/v0afQ6w3Bjw
youtu.be/v0afQ6w3Bjw
Reposted by John David Pressman
i think he's lost his juice and his senators are simply not afraid enough of him to follow this order. he's given it, what, three times now?
November 7, 2025 at 4:21 PM
i think he's lost his juice and his senators are simply not afraid enough of him to follow this order. he's given it, what, three times now?
Reposted by John David Pressman
Chatted with an OG Internet guy at the @atproto.boston meetup today.
Asked him about the last time he saw so much community excitement and engagement for a protocol.
He said http and html.
Asked him about the last time he saw so much community excitement and engagement for a protocol.
He said http and html.
November 7, 2025 at 4:50 AM
Chatted with an OG Internet guy at the @atproto.boston meetup today.
Asked him about the last time he saw so much community excitement and engagement for a protocol.
He said http and html.
Asked him about the last time he saw so much community excitement and engagement for a protocol.
He said http and html.
> I don't think the PDS should be doing mime type sniffing to begin with, but I don't know why this hasn't been fixed at the very least.
???
???
November 6, 2025 at 9:32 PM
> I don't think the PDS should be doing mime type sniffing to begin with, but I don't know why this hasn't been fixed at the very least.
???
???
Reposted by John David Pressman
for me, coming around on LLMs for learning and coding was less "i was wrong, these things are 100% accurate!" and more "wow, I really underestimated how many things are trivially fact-checkable but hard to surface"
November 6, 2025 at 9:20 AM
for me, coming around on LLMs for learning and coding was less "i was wrong, these things are 100% accurate!" and more "wow, I really underestimated how many things are trivially fact-checkable but hard to surface"
Reposted by John David Pressman
Market: Will Trump brag on TV at least three hours after he receives the FIFA peace prize at the World Cup draw?
manifold.markets/JohnDavidPre...
manifold.markets/JohnDavidPre...
Will Trump brag on TV at least three hours after he receives the FIFA peace prize at the World Cup draw?
59% chance. FIFA has announced it has created a peace prize which it will presumably give to Trump at the draw for the World Cup on December 5th:
https://bsky.app/profile/pbsnews.org/post/3m4wb3fc7j3...
manifold.markets
November 6, 2025 at 1:12 AM
Market: Will Trump brag on TV at least three hours after he receives the FIFA peace prize at the World Cup draw?
manifold.markets/JohnDavidPre...
manifold.markets/JohnDavidPre...