David Marx
@digthatdata.bsky.social
I like to post papers. They're not always new.
I read a lot: https://dmarx.github.io/papers-feed/
Complexity
Representation Learning
Agnatology, Epistemic Justice
Morality As Cooperation
Ontic Structural Realism
YIMBY, UBI
Research MLE
Former FireFighter
I read a lot: https://dmarx.github.io/papers-feed/
Complexity
Representation Learning
Agnatology, Epistemic Justice
Morality As Cooperation
Ontic Structural Realism
YIMBY, UBI
Research MLE
Former FireFighter
the DOJ wants you exhausted.
November 11, 2025 at 12:14 PM
the DOJ wants you exhausted.
i bet they all taste like salt
November 11, 2025 at 12:02 PM
i bet they all taste like salt
ensembling could also be worth experimenting with, might even be able to train an even smaller model arxiv.org/abs/2509.14786
Pre-training under infinite compute
Since compute grows much faster than web text available for language model pre-training, we ask how one should approach pre-training under fixed data and no compute constraints. We first show that exi...
arxiv.org
November 11, 2025 at 11:49 AM
ensembling could also be worth experimenting with, might even be able to train an even smaller model arxiv.org/abs/2509.14786
another thing you might consider playing with: squeeze more value from each datum with a diffusion objective arxiv.org/abs/2507.15857
Diffusion Beats Autoregressive in Data-Constrained Settings
Autoregressive (AR) models have long dominated the landscape of large language models, driving progress across a wide range of tasks. Recently, diffusion-based language models have emerged as a promis...
arxiv.org
November 11, 2025 at 11:45 AM
another thing you might consider playing with: squeeze more value from each datum with a diffusion objective arxiv.org/abs/2507.15857
did you compare against a TinyStories 50M?
November 11, 2025 at 4:36 AM
did you compare against a TinyStories 50M?
Reposted by David Marx
Along with Baguettotron we release the smallest viable language model to date. Monad, a 56M transformer, trained on the English part of SYNTH with non-random performance on MMLU. Desiging Monad an engineering challenge requiring a custom tiny tokenizer. huggingface.co/PleIAs/Monad
November 10, 2025 at 5:33 PM
Along with Baguettotron we release the smallest viable language model to date. Monad, a 56M transformer, trained on the English part of SYNTH with non-random performance on MMLU. Desiging Monad an engineering challenge requiring a custom tiny tokenizer. huggingface.co/PleIAs/Monad
Ran it by Claude: in addition to the section 213 shenanigans, the sec 116 "fiduciary relationship" carve out is weirdly specific. Claude suggested it might be designed to protect Andy Harris (R-MD) -- an anesthesiologist -- who prescribed Ivermectin to COVID-19 patients.
Andy Harris, congressman and anesthesiologist from Maryland, says he prescribed ivermectin for COVID
U.S. Rep. Andy Harris of Maryland, a practicing anesthesiologist, said on a radio show that he prescribed a medication typically used to treat parasites in livestock and humans as a treatment for C…
www.baltimoresun.com
November 11, 2025 at 4:28 AM
Ran it by Claude: in addition to the section 213 shenanigans, the sec 116 "fiduciary relationship" carve out is weirdly specific. Claude suggested it might be designed to protect Andy Harris (R-MD) -- an anesthesiologist -- who prescribed Ivermectin to COVID-19 patients.
"...but I will coordinate senators who aren't at imminent electoral risk to support the republican bill on my behalf, and then vote no and make statements like this one performatively."
November 10, 2025 at 7:52 PM
"...but I will coordinate senators who aren't at imminent electoral risk to support the republican bill on my behalf, and then vote no and make statements like this one performatively."
savage but accurate
November 10, 2025 at 7:32 PM
savage but accurate
congrats on successfully workshopping your joke sufficiently to permit delivering it without risk of confusion
November 10, 2025 at 7:28 PM
congrats on successfully workshopping your joke sufficiently to permit delivering it without risk of confusion
consider for example geocities and myspace
November 10, 2025 at 6:17 PM
consider for example geocities and myspace
i don't think that's a sustainable approach. personal sites churn. I find most links like this often don't work if they're even just 10 years old. one of the things that makes arxiv work is that it's a centralized hosting location. I think that ultimately needs to be a core part of it.
November 10, 2025 at 6:16 PM
i don't think that's a sustainable approach. personal sites churn. I find most links like this often don't work if they're even just 10 years old. one of the things that makes arxiv work is that it's a centralized hosting location. I think that ultimately needs to be a core part of it.
preprint -> postprint/socialmedia is an insightful take, thanks for sharing.
I like the idea of a general purpose public repository of academic content. if not arxiv, how might something like this work?
I like the idea of a general purpose public repository of academic content. if not arxiv, how might something like this work?
November 10, 2025 at 3:56 PM
preprint -> postprint/socialmedia is an insightful take, thanks for sharing.
I like the idea of a general purpose public repository of academic content. if not arxiv, how might something like this work?
I like the idea of a general purpose public repository of academic content. if not arxiv, how might something like this work?
he technically voted no, but we should prob add schumer to that list.
November 10, 2025 at 3:48 PM
he technically voted no, but we should prob add schumer to that list.
re text stuff specifically: you might find this interesting. bsky.app/profile/sung...
which extends Diffusion LLMs with the capabilities to remask, insert and delete tokens.
Paper: On Powerful Ways to Generate: Autoregression, Diffusion, and Beyond ( arxiv.org/abs/2510.06190 )
Repo: github.com/chr26195/AP-...
Paper: On Powerful Ways to Generate: Autoregression, Diffusion, and Beyond ( arxiv.org/abs/2510.06190 )
Repo: github.com/chr26195/AP-...
November 10, 2025 at 2:55 PM
re text stuff specifically: you might find this interesting. bsky.app/profile/sung...