This was a great collaboration with you @amandareillyinnz.bsky.social.

AI will do as it’s trained. If that is to squeeze more profit from workers at the cost of their health, it will. But if we train it to benefit the people whose life experiences make it work, then we will have AI for good.

This is going to end exactly like the google “glassholes” debacle. The crazy part is that we accept a degree of surveillance from smartphones in our own pockets that we would never accept from a surveillance device on someone else’s face.

"How could this happen?" Say the people who trained the AI to make this happen. The deniability is the point.

Gives new and bleak meaning to "dehumanizing", doesn't it.

It's not worth it. The damage to people is real and the system won't even create the savings they're training the AI to try to find.

4/4 ...then it will find every opportunity to eliminate the benefit, even where it is wrong. AI hallucinates. When the hidden optimization goal is reducing costs, AI hallucinates that people are abusing the system when they are not.

3/4 What's going wrong? The answer is hidden optimization goals. If an AI is told to make sure everyone who is eligible for a benefit gets it, it will do that. If it is told to kick everyone off of a benefit that it can, it will do that. If an AI has the hidden optimization goal of reducing costs...

2/4 And it worked out badly in India, on pretty much the same facts. Eligible people were kicked out without necessary benefits. www.amnesty.org/en/latest/ne...

1/4 This has worked out very badly for the UK, when they tried to hand over deciding who gets benefits or who should be sanctioned to AI. AI refused needed benefits to thousands of people who had no recourse in some cases, and showed bias in others. Example: www.theguardian.com/society/2024...