Lydia NZ
@zajirobo.bsky.social
Tippy-tappying on the computer | 🏳️🌈🇨🇦 living in 🇮🇪 | Free 🇵🇸 | No LGB without 🏳️⚧️ | she/her
"Let this be a lesson to you all...no one sex traffics on Facebook eighteen times in a row."
November 23, 2025 at 5:20 PM
"Let this be a lesson to you all...no one sex traffics on Facebook eighteen times in a row."
Impossible to pick just one. Marie Curie's frowny face in Radium Eyes always kills me
November 22, 2025 at 7:37 PM
Impossible to pick just one. Marie Curie's frowny face in Radium Eyes always kills me
@drewtoothpaste.bsky.social recently asked people to share their favourite TFD strips from the last 20 years and not a single other person said this one which makes it criminally underrated imo
November 22, 2025 at 4:05 PM
@drewtoothpaste.bsky.social recently asked people to share their favourite TFD strips from the last 20 years and not a single other person said this one which makes it criminally underrated imo
But overall I get what you're saying, and no I couldn't tell you which layers of an nn-Unet do edge detection. I guess it's how you choose to interpret the level of abstraction! Thanks for your thoughtful responses though, I appreciated reading them
November 4, 2025 at 3:56 PM
But overall I get what you're saying, and no I couldn't tell you which layers of an nn-Unet do edge detection. I guess it's how you choose to interpret the level of abstraction! Thanks for your thoughtful responses though, I appreciated reading them
"Black box" is one thing, but "unknowable construct that can't be described by mathematical concepts" is too far a step from what is tensor multiplication for me.
I feel like there is lots of work on explainability and doing exactly what you say in trying to understand which layers do what etc.
I feel like there is lots of work on explainability and doing exactly what you say in trying to understand which layers do what etc.
November 4, 2025 at 3:56 PM
"Black box" is one thing, but "unknowable construct that can't be described by mathematical concepts" is too far a step from what is tensor multiplication for me.
I feel like there is lots of work on explainability and doing exactly what you say in trying to understand which layers do what etc.
I feel like there is lots of work on explainability and doing exactly what you say in trying to understand which layers do what etc.
Sure, I can't physically do the math by hand, it would take me literal ages. Does that mean I don't know what an nnU-Net is doing or what it fundamentally is? This is the part I disagree with and would love to understand what I'm not getting.
November 4, 2025 at 3:33 PM
Sure, I can't physically do the math by hand, it would take me literal ages. Does that mean I don't know what an nnU-Net is doing or what it fundamentally is? This is the part I disagree with and would love to understand what I'm not getting.
I work with nnU-Nets, they are laughably small in comparison, but I *define* the number of layers, the input size and shape, the number of trainable parameters etc. and then I can load an actual model file. How are huge LLMs fundamentally different from this, architecturally speaking?
November 4, 2025 at 3:33 PM
I work with nnU-Nets, they are laughably small in comparison, but I *define* the number of layers, the input size and shape, the number of trainable parameters etc. and then I can load an actual model file. How are huge LLMs fundamentally different from this, architecturally speaking?
We had to know this to be able to initially define it and train it. Is the model somehow modifying its own underlying architecture while being trained? Yes its actual numerical and computational complexity is beyond our grasp but saying it's become some unknowable *thing* sounds like hype to me?
November 4, 2025 at 3:33 PM
We had to know this to be able to initially define it and train it. Is the model somehow modifying its own underlying architecture while being trained? Yes its actual numerical and computational complexity is beyond our grasp but saying it's become some unknowable *thing* sounds like hype to me?
Would you mind pointing me to some of the research? I'd love to have a closer look. But what I can't seem to get past is, fundamentally most big LLMs are transformer models. We know exactly what a transformer model is in terms of layers, convolutions, pooling etc.
November 4, 2025 at 3:33 PM
Would you mind pointing me to some of the research? I'd love to have a closer look. But what I can't seem to get past is, fundamentally most big LLMs are transformer models. We know exactly what a transformer model is in terms of layers, convolutions, pooling etc.
Given the knowledge gaps in how we learn and understand OUR OWN language(s) as a species, confidently saying "this mathematical model also uses language the same way we do!" is...a reach! Which is all I was saying. Apologies for not being more clear but like...these are pretty low stakes here dude
November 4, 2025 at 3:07 PM
Given the knowledge gaps in how we learn and understand OUR OWN language(s) as a species, confidently saying "this mathematical model also uses language the same way we do!" is...a reach! Which is all I was saying. Apologies for not being more clear but like...these are pretty low stakes here dude
What? I merely brought it up as "hey isn't this weird? This unfortunate person was not spoken to like every other human being was at a developmentally critical period of her life, and she was never able to learn language as a result and we don't know exactly why. Isn't that weird? Anyways".
November 4, 2025 at 3:03 PM
What? I merely brought it up as "hey isn't this weird? This unfortunate person was not spoken to like every other human being was at a developmentally critical period of her life, and she was never able to learn language as a result and we don't know exactly why. Isn't that weird? Anyways".
more like uncanny mimicry than true understanding. But really cool to see how these funny problems/errors in parsing actually lead to further technical understanding and development. Thanks for pointing that out!
November 4, 2025 at 2:56 PM
more like uncanny mimicry than true understanding. But really cool to see how these funny problems/errors in parsing actually lead to further technical understanding and development. Thanks for pointing that out!
Yeah, I'm not saying "this one anecdote is evidence it will never work and it's garbage" and I understand the technical implications of why it makes this mistake. I'm just saying there are clearly gaps, and if you're being skeptical (which is fine! The stakes are actually quite low here) it looks
November 4, 2025 at 2:56 PM
Yeah, I'm not saying "this one anecdote is evidence it will never work and it's garbage" and I understand the technical implications of why it makes this mistake. I'm just saying there are clearly gaps, and if you're being skeptical (which is fine! The stakes are actually quite low here) it looks
a long way to go before I'm convinced, which is fine? Not sure why you're taking this so personally.
November 4, 2025 at 2:52 PM
a long way to go before I'm convinced, which is fine? Not sure why you're taking this so personally.
I was enjoying this good faith discussion with everyone else but you've kind of taken it off the rails here. I never said anything about human superiority, I said I was skeptical that a mathematical language model can reason or understand. Chain of thought models are cool, but there's still
November 4, 2025 at 2:52 PM
I was enjoying this good faith discussion with everyone else but you've kind of taken it off the rails here. I never said anything about human superiority, I said I was skeptical that a mathematical language model can reason or understand. Chain of thought models are cool, but there's still
can falsely convince us that they understand and are communicating *in the same way* with the same meanings assigned to the words themselves, and maybe that's where the misunderstanding/danger is. Just my two cents though. Also brings to mind Koko the gorilla, Travis the chimpanzee etc.
November 4, 2025 at 2:42 PM
can falsely convince us that they understand and are communicating *in the same way* with the same meanings assigned to the words themselves, and maybe that's where the misunderstanding/danger is. Just my two cents though. Also brings to mind Koko the gorilla, Travis the chimpanzee etc.
I think (and this is just my opinion, I obviously didn't write it) the authors used the word "parrot" not as "parrots are dumb but can trick us to think they're smart because they can mimic us, therefore LLMs are dumb too". I think it's more subtle than that - their uncannily accurate mimicry
November 4, 2025 at 2:42 PM
I think (and this is just my opinion, I obviously didn't write it) the authors used the word "parrot" not as "parrots are dumb but can trick us to think they're smart because they can mimic us, therefore LLMs are dumb too". I think it's more subtle than that - their uncannily accurate mimicry
You are reading a lot into what I am saying. We don't fully understand our own sentience, language is complex, and being skeptical of claims of understanding by a model that can mimic human speech but can't tell how many "r"s are in the word strawberry doesn't make me anti AI, it makes me skeptical.
November 4, 2025 at 2:34 PM
You are reading a lot into what I am saying. We don't fully understand our own sentience, language is complex, and being skeptical of claims of understanding by a model that can mimic human speech but can't tell how many "r"s are in the word strawberry doesn't make me anti AI, it makes me skeptical.
Oh cool, thanks for this. Will read this when I have a chance
November 4, 2025 at 2:13 PM
Oh cool, thanks for this. Will read this when I have a chance
but how is this different from the fact that we no longer are able to fully grasp every line of the resulting assembly code of modern compiled executables? Or the transistor-level logic of a PC running Windows? Does it matter?
November 4, 2025 at 2:12 PM
but how is this different from the fact that we no longer are able to fully grasp every line of the resulting assembly code of modern compiled executables? Or the transistor-level logic of a PC running Windows? Does it matter?
But surely we understand the fundamental blocks of the model, because how else would it be deployed? We understand at a high level what each of the layers are doing, how the inputs are concatenated. Sure we don't have a good grasp of what 11 billion trainable parameters looks like as a spreadsheet,
November 4, 2025 at 2:12 PM
But surely we understand the fundamental blocks of the model, because how else would it be deployed? We understand at a high level what each of the layers are doing, how the inputs are concatenated. Sure we don't have a good grasp of what 11 billion trainable parameters looks like as a spreadsheet,
As far as I am aware there is something fundamental that we do not understand about language that must be learned at a very early age otherwise it cannot be replicated, despite the fact that she could communicate and learned many words - she was never able to fully replicate human speech.
November 4, 2025 at 2:07 PM
As far as I am aware there is something fundamental that we do not understand about language that must be learned at a very early age otherwise it cannot be replicated, despite the fact that she could communicate and learned many words - she was never able to fully replicate human speech.
despite being rescued and taken into care. She was looked after by a number of linguists (and even lived with some of them) but it was the 1970s and the ethics were questionable at best, and likely did not have her best interests at heart.
November 4, 2025 at 2:07 PM
despite being rescued and taken into care. She was looked after by a number of linguists (and even lived with some of them) but it was the 1970s and the ethics were questionable at best, and likely did not have her best interests at heart.
This reminds me of the obviously flawed study of Genie, a "feral child" who was horribly abused by her father and as a result did not learn language. This is based only on my reading of her Wikipedia article, but as I understood it she was never able to fully develop language including grammar
November 4, 2025 at 2:07 PM
This reminds me of the obviously flawed study of Genie, a "feral child" who was horribly abused by her father and as a result did not learn language. This is based only on my reading of her Wikipedia article, but as I understood it she was never able to fully develop language including grammar
I am not an expert though so would love to have these debates (in good faith obviously)
November 4, 2025 at 2:01 PM
I am not an expert though so would love to have these debates (in good faith obviously)