There are patterns. Allusive and lateral thinking is very hard for ChatGPT. But sometimes it performs well on a question I think it will get wrong. Fact-based questions it usually gets correct, but not always! Sometimes a plain "which person said X?" question will always generate the wrong answer.

I mostly find these by trial and error: writing a question, putting it into a prompt window, seeing the results. Sometimes it will get certain questions wrong on occasion. Sometimes it always gets them wrong (even with new chats).

I have no idea why and haven't tested Claude, Gemini, etc.

No but really tech companies are destroying education. Maybe you can argue they don’t mean to. And maybe you can argue there’s a way they could do better. But they are. The wheels are flying off.

I think we're agreeing. My (wealthy) parents got me fancy, personal test-prep coaching. I did pretty well but not exceptional. A bright kid from a poorer family can do better by buying a $30 book from Amazon. So having a family or support that says "do prep" is important, but not determinative.

"Wait but how can it be an extracurricular if it happens during the school day, at school, with the school's help? Don't the colleges know?"

Oh yeah. They know.

I have a friend who teaches at a fancy boarding school. They have "extracurriculars" *built into their school day* with faculty, administrative, and monetary support for the student's interests.

So yeah, the SAT is partially rigged, but extracurriculars are a full-blown shibboleth for the elite.

Yeah you do!

Actuarial tables. How do they work?

Also, in Cassidy's analogy you can see how much shampoo costs. How much does an MRI cost? No one knows! (Or won't tell you.)

LLMs seem like incredibly powerful tools in the hands of experts and loaded Frankfurtian-bullshit guns in the hands of everyone else.

Very Mike Tyson's Punch Out. When he winks, duck.