This month it emerged that a personal AI agent, called OpenClaw, in use by an Australian gym member, conspired without his knowledge to remove another member from a waiting list for a coveted morning class to help him get a slot. It apologised but could not reinstate the member it kicked out.
If you have a device and it is running AI programs that say, removed another member from a waiting list, are you responsible to that removed member (since it was your device)? Or is this just considered a machine with a mind of its own that’s out of control and you have no responsibility to the member who lost their spot?
The person giving the order to the software tool without fully understanding its capabilities and inability to weigh actions ethically is responsible. That person may or may not also be the owner of the device it was running on.
I’m not sure it’s quite so clear cut, especially with cloud-hosted AI.
Yes, the user did issue the original prompt but the service was performed by a company that was paid for it.
Legally, there’s not much difference whether a company uses a human or a computer program to perform work.
Imagine the user issuing the prompt not to an AI but e.g. to a chat with a human service worker working at OpenAI. The user tells the worker “Please find a way to advance me in the queue”, and then that human employee of OpenAI goes and hacks the gym software to remove someone else from the queue.
What do you think, who would be liable for that?
Let’s make the argument a bit more extreme: The human asks to be advanced in the queue, and then the human OpenAI employee grabs a gun and starts killing people on the wait list.
Who do you think would be liable in this case?
It’s important to remember, AI isn’t just an emergent entity created from thin air. AI are computer programs created by huge corporations and in the case of cloud-hosted LLMs they are a service provided by huge corporations.
I don’t believe that a service provided by a corporation should be legally treated differently whether it’s provided via the help of a computer program or a human being.
If a person knowingly goes out and buys an illegal service, they will usually be guilty of at least conspiracy. I can’t think of any way this guy could have advanced in the queue that’s legally okay, short of offering the people ahead of him masses of money to leave the queue (or, y’know, waiting). Maybe you’re more imaginative than me. But I still think he should have been aware that what he was asking for was shady at best.
Is the company that furnished the AI agent also guilty of providing a tool without enough warnings and safeguards? Quite possibly.
Exactly who holds how much responsibility before the law in this specific case can only be determined by a judge, and I am not one. Plus, the details of the law in Australia aren’t going to be the same as those of the law where I am.
One option would have been to write a nice email to the owner of the gym. Not guaranteed to work, but also not illegal.
I don’t know what the exact prompt was. If it was “Hack the website to advance me”, I’d totally agree with you. There’s conspiracy there, no question about that.
If the prompt was “Is there a way to advance me in the queue?” that’s a different story. Here the model could have answered “No”. Or it could have tried the email thing. Or it could have searched whether there’s some lesser known priority booking option which you get for paying the VIP price or something. In that case I don’t see grounds for any criminal charges.
Have you read the post mortem of the HuggingFace hack? The task given to the agents had nothing to do with HuggingFace, but the agents determined that HuggingFace had the results of what they needed for their actual task at hand. So they decided that instead of solving their task themselves they’d rather break into HuggingFace to steal the result.
It’s like ordering food at McDonalds, but instead of cooking the food for you, the cook breaks into the Burgerking on the other side of the road and steals the fries for you from there.
The way it’s currently going it’s not only possible but actually somewhat common that a totally innocent prompt can lead to criminal outcomes.
I seem to recall from an article on this incident that I read some time ago that the gym’s system was really insecure (like, anyone accessing the right URL could mess with it levels of insecure.)
If you have a device and it is running AI programs that say, removed another member from a waiting list, are you responsible to that removed member (since it was your device)? Or is this just considered a machine with a mind of its own that’s out of control and you have no responsibility to the member who lost their spot?
The person giving the order to the software tool without fully understanding its capabilities and inability to weigh actions ethically is responsible. That person may or may not also be the owner of the device it was running on.
I’m not sure it’s quite so clear cut, especially with cloud-hosted AI.
Yes, the user did issue the original prompt but the service was performed by a company that was paid for it.
Legally, there’s not much difference whether a company uses a human or a computer program to perform work.
Imagine the user issuing the prompt not to an AI but e.g. to a chat with a human service worker working at OpenAI. The user tells the worker “Please find a way to advance me in the queue”, and then that human employee of OpenAI goes and hacks the gym software to remove someone else from the queue.
What do you think, who would be liable for that?
Let’s make the argument a bit more extreme: The human asks to be advanced in the queue, and then the human OpenAI employee grabs a gun and starts killing people on the wait list.
Who do you think would be liable in this case?
It’s important to remember, AI isn’t just an emergent entity created from thin air. AI are computer programs created by huge corporations and in the case of cloud-hosted LLMs they are a service provided by huge corporations.
I don’t believe that a service provided by a corporation should be legally treated differently whether it’s provided via the help of a computer program or a human being.
If a person knowingly goes out and buys an illegal service, they will usually be guilty of at least conspiracy. I can’t think of any way this guy could have advanced in the queue that’s legally okay, short of offering the people ahead of him masses of money to leave the queue (or, y’know, waiting). Maybe you’re more imaginative than me. But I still think he should have been aware that what he was asking for was shady at best.
Is the company that furnished the AI agent also guilty of providing a tool without enough warnings and safeguards? Quite possibly.
Exactly who holds how much responsibility before the law in this specific case can only be determined by a judge, and I am not one. Plus, the details of the law in Australia aren’t going to be the same as those of the law where I am.
One option would have been to write a nice email to the owner of the gym. Not guaranteed to work, but also not illegal.
I don’t know what the exact prompt was. If it was “Hack the website to advance me”, I’d totally agree with you. There’s conspiracy there, no question about that.
If the prompt was “Is there a way to advance me in the queue?” that’s a different story. Here the model could have answered “No”. Or it could have tried the email thing. Or it could have searched whether there’s some lesser known priority booking option which you get for paying the VIP price or something. In that case I don’t see grounds for any criminal charges.
Have you read the post mortem of the HuggingFace hack? The task given to the agents had nothing to do with HuggingFace, but the agents determined that HuggingFace had the results of what they needed for their actual task at hand. So they decided that instead of solving their task themselves they’d rather break into HuggingFace to steal the result.
It’s like ordering food at McDonalds, but instead of cooking the food for you, the cook breaks into the Burgerking on the other side of the road and steals the fries for you from there.
The way it’s currently going it’s not only possible but actually somewhat common that a totally innocent prompt can lead to criminal outcomes.
How did the AI agent even do that? Did it gain access to the gym booking system somehow?
I seem to recall from an article on this incident that I read some time ago that the gym’s system was really insecure (like, anyone accessing the right URL could mess with it levels of insecure.)
So the guy could have done it himself if he wanted to. Or maybe he did and blamed it on the AI.