AI does quality review too. Imperfectly,
but the individual functions of AI acting within those systems have all improved dramatically in the past 12 months
I suppose that depends on how we define improvement, because from where I’m sitting, it’s reasonable to be apprehensive about LLM’s and their output when we see spectacular failure after spectacular failure.
Whether it’s bombing a school in Iran because Claude fucked up the targeting, or an AI agent deleting your email inbox or your production database, or creating a court case out of thin air, or stats in a SCOTUS ruling that are fictitious, over and over and over again the extravagant promises they keep telling us are just around the corner appear to be decidedly half-baked.
And if you use Teams or Windows and pieces of functionality that worked for two decades are no longer working as designed in a dependable way, I guess I just don’t know what to tell you.
It makes perfect sense not to trust this technology, and the speed it promises is often mitigated by the fact that you can’t and shouldn’t trust its output, because if you’re the unlucky SOB that doesn’t check a reference, you can literally become national news.
Further, being that it’s already been trained on the entirety of recorded human knowledge, I’m not sure how it gets better either. You can make it faster, but it’s just going to spit out slop at a faster rate.
Whether it’s bombing a school in Iran because Claude fucked up the targeting
I’m going to call user error on that, and I don’t think it matters what system they were using - they were going to make mistakes.
an AI agent deleting your email inbox or your production database
The real error there? Conducting risky operations without backups.
creating a court case out of thin air
That’s just big silicon-brass balls. Interns do it too, but you don’t hear about them. On the other hand, trusting the AI or the intern, that’s disbarment levels of reckless.
It makes perfect sense not to trust this technology
Or any technology, until we have figured out what it is, and isn’t, capable of doing reliably.
But, plenty of people still play Russian Roulette, for one reason or another. Is that the revolver manufacturer’s fault?
being that it’s already been trained on the entirety of recorded human knowledge, I’m not sure how it gets better either
Yes, you can mitigate any risk you can think of with the right planning, but these are new risks of arbitrary severity, from trivial to devastating, that did not exist at all before. Previous systems had risks, but they were different, more limited in scope, and accounted for.
It may be true that some of these systems are worth those new risks and the planning necessary to find and mitigate them all, but we have to do that hard work and be real with ourselves, rather than hand-wave them away because of the potentially thrilling prospects.
but these are new risks of arbitrary severity, from trivial to devastating, that did not exist at all before
I disagree. The risks of arbitrary severity are already there, we have just slowly evolved into handling them how we handle them now, learned (sometimes) from the mistakes either how to handle them better or just accept the consequences as “how things are.”
So, I do agree that rapid switching to letting AI handle things will mis-handle the old risks in new and unanticipated ways. Maybe if AI helps us actually build a space elevator or functional fusion Tokamaks or other things like that then that will be creating significant new risks with their own new issues.
we have to do that hard work and be real with ourselves, rather than hand-wave them away because of the potentially thrilling prospects.
AI is just the latest shiny in a long list of shinies that enthusiastic entreprenuers have hand-waved all the hard work around the real challenges away about. I think self driving cars are a pretty good example of how this “disaster waiting to happen” will really roll out - things like https://www.youtube.com/watch?v=BYkv6jvTpCchttps://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg etc. Is it regrettable that Elaine died? Yes - right along with the other 6000+ pedestrians killed by human drivers that year in the US alone.
I suppose that depends on how we define improvement, because from where I’m sitting, it’s reasonable to be apprehensive about LLM’s and their output when we see spectacular failure after spectacular failure.
Whether it’s bombing a school in Iran because Claude fucked up the targeting, or an AI agent deleting your email inbox or your production database, or creating a court case out of thin air, or stats in a SCOTUS ruling that are fictitious, over and over and over again the extravagant promises they keep telling us are just around the corner appear to be decidedly half-baked.
And if you use Teams or Windows and pieces of functionality that worked for two decades are no longer working as designed in a dependable way, I guess I just don’t know what to tell you.
It makes perfect sense not to trust this technology, and the speed it promises is often mitigated by the fact that you can’t and shouldn’t trust its output, because if you’re the unlucky SOB that doesn’t check a reference, you can literally become national news.
Further, being that it’s already been trained on the entirety of recorded human knowledge, I’m not sure how it gets better either. You can make it faster, but it’s just going to spit out slop at a faster rate.
I’m going to call user error on that, and I don’t think it matters what system they were using - they were going to make mistakes.
The real error there? Conducting risky operations without backups.
That’s just big silicon-brass balls. Interns do it too, but you don’t hear about them. On the other hand, trusting the AI or the intern, that’s disbarment levels of reckless.
Or any technology, until we have figured out what it is, and isn’t, capable of doing reliably.
But, plenty of people still play Russian Roulette, for one reason or another. Is that the revolver manufacturer’s fault?
Better editing.
Yes, you can mitigate any risk you can think of with the right planning, but these are new risks of arbitrary severity, from trivial to devastating, that did not exist at all before. Previous systems had risks, but they were different, more limited in scope, and accounted for.
It may be true that some of these systems are worth those new risks and the planning necessary to find and mitigate them all, but we have to do that hard work and be real with ourselves, rather than hand-wave them away because of the potentially thrilling prospects.
I disagree. The risks of arbitrary severity are already there, we have just slowly evolved into handling them how we handle them now, learned (sometimes) from the mistakes either how to handle them better or just accept the consequences as “how things are.”
So, I do agree that rapid switching to letting AI handle things will mis-handle the old risks in new and unanticipated ways. Maybe if AI helps us actually build a space elevator or functional fusion Tokamaks or other things like that then that will be creating significant new risks with their own new issues.
AI is just the latest shiny in a long list of shinies that enthusiastic entreprenuers have hand-waved all the hard work around the real challenges away about. I think self driving cars are a pretty good example of how this “disaster waiting to happen” will really roll out - things like https://www.youtube.com/watch?v=BYkv6jvTpCc https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg etc. Is it regrettable that Elaine died? Yes - right along with the other 6000+ pedestrians killed by human drivers that year in the US alone.