Although there are a few ways to mitigate the risk, the only way to block it is to get AI to differentiate instructions from data, which is impossible today.
I have heard in the past that it’s not possible to fully control AI. Like literally, the people developing and running the AI cannot fully control its behavior. I did a quick search to see if I could find more info and found this link on the first page of results: https://www.eurekalert.org/news-releases/1032090
I think that we’re going to continue seeing unwanted behavior from AI.
if the instruction is messy fuzzy human language to a system that was not coded instruction by instruction but got generated and trained then there never is a way to differentiate instructions from data if i’m not mistaken
We know that some humans can be trained to do that just fine. Humans are natural neuronal networks. That implies, neuronal networks can in principle do it. We just don’t have any human-capability artificial neuronal networks yet.
LLMs might never get there. But humans aren’t LLMs. If we ever manage to properly model a human brain, that probably will be able to do that task with human-level accuracy (which actually is pretty good if you only look at professionals of the filed).
Hopefully, it doesn’t actually need a human brain for the task - because modeling that might still be a century off.
It is - and tons of bugs are just about fucking it up or just not doing it at all. Incomplete or faulty parsing and validation of untrusted input is absurdly common. It got better in the past decades, but most software developers write worse code than AI by now (not me though; obviously, I am still better than the best AI).
Input sanitation, basically security 101. And it can’t currently do it…
I have heard in the past that it’s not possible to fully control AI. Like literally, the people developing and running the AI cannot fully control its behavior. I did a quick search to see if I could find more info and found this link on the first page of results: https://www.eurekalert.org/news-releases/1032090
I think that we’re going to continue seeing unwanted behavior from AI.
That’s also not the only way. Basic governance also works. Why does copilot have so many permissions?
Little Bobby Tables strikes again.
if the instruction is messy fuzzy human language to a system that was not coded instruction by instruction but got generated and trained then there never is a way to differentiate instructions from data if i’m not mistaken
Well, good thing we’ve only poured a trillion and a half dollars into it and wrecked the economy.
We know that some humans can be trained to do that just fine. Humans are natural neuronal networks. That implies, neuronal networks can in principle do it. We just don’t have any human-capability artificial neuronal networks yet.
LLMs might never get there. But humans aren’t LLMs. If we ever manage to properly model a human brain, that probably will be able to do that task with human-level accuracy (which actually is pretty good if you only look at professionals of the filed).
Hopefully, it doesn’t actually need a human brain for the task - because modeling that might still be a century off.
I just can’t anymore. Isn’t that like, the basic thing any program does? Who runs these companies?
It is - and tons of bugs are just about fucking it up or just not doing it at all. Incomplete or faulty parsing and validation of untrusted input is absurdly common. It got better in the past decades, but most software developers write worse code than AI by now (not me though; obviously, I am still better than the best AI).