I think you're right though, teaching to the why might be key here. There are better ways to show that AI is untrustworthy than just trying to catch it in a gotcha moment - like showing how genAI formulates its answers by mashing language together, and how it makes up sources.

I'm just speaking from my experience, maybe OPs application was more successful. But if I did that assignment 5 times, and found AI was wrong >20% of the time, I'd still be more likely to trust AI and not fact check it in the future. It's risky, esp as gAI gets more updates.

I think its more important to teach to sources. Every piece of info has a source, someone who originally came up with the idea. If you can't find a source for the info, then the info should be suspect, because we should be able to trace the idea's ideological course through history.

I'm a luddite at heart, but for kids who like AI, this assignment could convince them that teachers/adults are exaggerating the claims that AI is untrustworthy. Gen AI is wrong a lot, but its not ALWAYS wrong, so don't expect that you're always going to find a "glue in the pizza" moment.