What if someone is obviously out of their mind and says yes? Random example, say someone has a stroke and has just barely figured out how to use the big red emergency call button on their phone screen. The AI asks if they are calling about blah blah emergency. Maybe the person only understands half of it, thinks they’re being asked if they’re calling about an emergency and say “yes”. Now they’re stuck having a form letter read to them by AI while their brain is dying. A person would catch something was up even if they said yes.
The article even notes this can be a problem without anything medical going on:
It’s also possible that AI agents handling 911 triage could fail to understand individuals with stronger accents and dialects, as well as differences in pitch and articulation. This is because AI is programmed with automatic speech recognition systems and, therefore, could potentialy be unreliable when presented with a speech unlike the ones used the AI is trained to recognize.
Responding to people having emergencies is too important a task to leave up to voice recognition slapped on top of an LLM.
I don’t know the concrete answer to your "what if,’ and I don’t think you or the author of the article do either. That would be a question for Carbyne.
I would speculate that there is an answer, and that neither you nor the author are the first to think of these problem scenarios. There’s design, engineering, and real-world testing involved.
What if someone is obviously out of their mind and says yes? Random example, say someone has a stroke and has just barely figured out how to use the big red emergency call button on their phone screen. The AI asks if they are calling about blah blah emergency. Maybe the person only understands half of it, thinks they’re being asked if they’re calling about an emergency and say “yes”. Now they’re stuck having a form letter read to them by AI while their brain is dying. A person would catch something was up even if they said yes.
The article even notes this can be a problem without anything medical going on:
Responding to people having emergencies is too important a task to leave up to voice recognition slapped on top of an LLM.
I don’t know the concrete answer to your "what if,’ and I don’t think you or the author of the article do either. That would be a question for Carbyne.
I would speculate that there is an answer, and that neither you nor the author are the first to think of these problem scenarios. There’s design, engineering, and real-world testing involved.
“Well I just trust they thought of it” isn’t good enough.
If you’re unsatisfied with my answer, then maybe consider asking them. I’m just a guy writing comments on Lemmy.