"better than chance" is how all scientific discovery is conducted. Therapy is better than chance, direct instruction is better than chance, medicine is better than chance. These are important ways to measure how and when to deploy technology. It means better than baseline, or control - so a positive

Depends on what you put into net-value, this seems like a positive thing. And does mean that we could use llms as learning aid with a real effect. It can absolutely be used as a positive. These numbers are good—it is almost a standard derivation. It isn't a zero sum game. Why not both?

Now these studies, show a positive effect which is what I noted—i am not sure why you are adding the stipulation that it has to be as good as a human, you said nothing of value and clearly these systems do aid in learning. We can have both LLMs and teachers. They clearly show value.

Further the authors note that their findings are in line with this meta-analysis, however it is one I have not read. www.sciencedirect.com/science/arti...

Whether or not text is predictive doesn't mean that it can't be useful for learning.

www.nature.com/articles/s41...

Meta-analysis, n=54 studies, with a very rigorous exclusion criteria. Included studies had to be experimental and include a control group. Most education science is never replicated unfortunately, but meta-analysis are still some of the best studies you can get.

LLMs are great tree search engines for scientific discovery - alpha evolve is one such system. They are also great teachers, which people seem to conviently ignore. A meta-analysis of N=54 studies, found that chatGPT aided in teaching - when constrained - with g=0.874, almsot a standard derivation.

If you ever read any paper that involves students, all you have to ask is: "What did they make them do?" - half of the time the entire paper crumbles.

This paper:
"What did they make them do?"
"Copy paste from chatGPT for 20 bucks a session."
Wow! I wonder why they werent engaged!?

"No it’s literally causing our brains to atrophy."
I am saying that the study does not prove this, and does not make your future predictions true. It measures boredom.

I am not dishonest; I responded to the original claim. The study measures boredom over 20x4 minutes, and is not definitive proof of what you say. It also has a finding that contradicts it. IE that you can use AI to aid in learning and enagement.

That is not how neurons work. Your brain does not atrophy if you are bored for 80 minutes. That is what the study is measuring. The EEG is measuring the boredom of n=18 students, this is not proof that your brain atrophies. Again, discussing with an LLM after writing shows more engagement.

The study states that the brain-only group had MORE engagement when using LLMs in the fourth session. That is a finding that you are ignoring. I will agree that not praciticing a skill makes you worse at it, but I need to remind you of your original claim: "Atrophies your brain". It does not do this

Because I read the article, and the study. They tested students on whether copy-pasting from LLMs made them less likely to be engaged/lazy - and paid them to do so. You are saying this atrophies the brain. I am saying that it doesn't measure that. I provided proof from your article that it doesn't.

I am not cherry picking, the study encouraged copy-pasting from chatGPT. I don't think I need to tell you that if I copy pasted from a badly written article my writing would be worse.

You are ignoring the finding that engagement with LLMs does not infact atrophy your brain.

The brain only group on their fourth session had more brain engagement: "The second group, in contrast, performed well, exhibiting a significant increase in brain connectivity across all EEG frequency bands."

But the study also claims the contrary that engaging with AI after having done work engages the brain more - if you trust EEG to measure what you think its measuring then surely that is a claim that is empowering for the AI defenders here? That is to say there are ways to use LLMs smartly.

Have you read the study you are citing? This study measures the generation effect which is not unique to LLMs. They paid students to write SAT essays, and where quite surprised that when you had the ability to skip that work and still walk home with 20 bucks, you weren't as engaged.