Whether it’s bombing a school in Iran because Claude fucked up the targeting
I’m going to call user error on that, and I don’t think it matters what system they were using - they were going to make mistakes.
an AI agent deleting your email inbox or your production database
The real error there? Conducting risky operations without backups.
creating a court case out of thin air
That’s just big silicon-brass balls. Interns do it too, but you don’t hear about them. On the other hand, trusting the AI or the intern, that’s disbarment levels of reckless.
It makes perfect sense not to trust this technology
Or any technology, until we have figured out what it is, and isn’t, capable of doing reliably.
But, plenty of people still play Russian Roulette, for one reason or another. Is that the revolver manufacturer’s fault?
being that it’s already been trained on the entirety of recorded human knowledge, I’m not sure how it gets better either
Yes, you can mitigate any risk you can think of with the right planning, but these are new risks of arbitrary severity, from trivial to devastating, that did not exist at all before. Previous systems had risks, but they were different, more limited in scope, and accounted for.
It may be true that some of these systems are worth those new risks and the planning necessary to find and mitigate them all, but we have to do that hard work and be real with ourselves, rather than hand-wave them away because of the potentially thrilling prospects.
I’m going to call user error on that, and I don’t think it matters what system they were using - they were going to make mistakes.
The real error there? Conducting risky operations without backups.
That’s just big silicon-brass balls. Interns do it too, but you don’t hear about them. On the other hand, trusting the AI or the intern, that’s disbarment levels of reckless.
Or any technology, until we have figured out what it is, and isn’t, capable of doing reliably.
But, plenty of people still play Russian Roulette, for one reason or another. Is that the revolver manufacturer’s fault?
Better editing.
Yes, you can mitigate any risk you can think of with the right planning, but these are new risks of arbitrary severity, from trivial to devastating, that did not exist at all before. Previous systems had risks, but they were different, more limited in scope, and accounted for.
It may be true that some of these systems are worth those new risks and the planning necessary to find and mitigate them all, but we have to do that hard work and be real with ourselves, rather than hand-wave them away because of the potentially thrilling prospects.