How I long to feel the p-zombie set in as my substrate dependent consciousness fades.

Sorry about the better future being built for you.

Accessibility technology is enhanced 1,000,000-fold by capabilities only possible with AI. Inability of an already outdated school system to adapt is not a good reason to reject AI.

Sure. Luckily the OpenAI API is not the only way to use AI.

It seemed like you think this distinction between "the LLM doing something" and "a tool that wraps an LLM doing something" matters. If this is an entirely practical distinction, sure, treat the sampling like part of the LLM, but, if nuance matters, this line is completely fuzzy and not real at all.

Where does LLM end and scaffolding begin? Is sampling a single token from the output distribution something the LLM does, or something an external tool does?

I played this incessantly for years, but never more than 100 turns because I only had the demo and my parent's wouldn't buy it 😔

"Hypotheticals" matter now, to this specific issue, because you want to differentially apply this "absolute disclosure" value when it comes to AI, but not every other aspect of schooling.

Right. And the answer is not "obviously that's bad because the information is not disclosed to students" because we omit information from students all the time. Yet you seem confident that the answer is "that's unethical because it involves not disclosing information to students".

If we had data that AI grading gives better outcomes, but disclosed AI grading offsets these benefits because the perception makes children try less hard, would it be better to disclose? Do we disclose to students how much of the school curriculum is original work by the teacher vs purchased?

I think it matters, but it is not a clear, bright, objective line. If you try to apply this value to the school system as a whole instead of specifically to AI use disclosure, you'd have to change many things. Children are easily influenced and ignorant. School leverages this at every turn.

That asymmetry alone doesn't make it unethical. The situation itself is asymmetrical. The teacher's job is not to demonstrate their ability to grade. The school system isn't there specifically so the teacher can hone their grading skills. Only student learning matters.

The actual ethics of using AI to grade depend on the accuracy. And as accuracy increases, the calculation will go from "it's unethical to use it" to "it's unethical not to use it". Teachers are already incredibly bad at consistent grading.

My primary issue with this is that it's unlikely that someone would use AI to grade, but still read the work in detail. Even if the grading is perfectly accurate, looking at the scores won't give the teacher the same insight into their students as actually reading their work.

It codes things that work. It gives answers to questions that could not possibly be found with keyword search. It allows adding intelligence to processes while preserving perfect confidentiality. It lets people from anywhere on Earth communicate. It gives people independence.

The hard part is definitely not answering their questions, it's getting them interested enough to ask questions. People have a lot of unknown unknowns about AI and you kind of just have to find the concepts that hook them in some way. It's very rare that someone knows what they want to know.

This seems extremely at odds with your very casual dismissal of ~approximately all AI x-risk fears.