Do you honestly think that nobody has thought about that and solved that problem already? Even with all the engineers involved, and after all the real-world testing, you’re the first to have considered that scenario?
Even consumer-grade products like ChatGPT can be interrupted while talking. We’re talking about an AI implementation, not some rigid set of if...else statements.
Do you honestly think that nobody has thought about that and solved that problem already? Even with all the engineers involved, and after all the real-world testing,
That is the thought process of a child. If your defense of something with obvious issues is “Well I’m sure they know what they’re doing, because if they didn’t it would be bad”, I have some really, really bad news for you about the world.
If this was thought through by a capable adult, it would’t be implemented at all. The very fact that this solution is on the table is proof of the breakdown of rational thought. Or more likely, greased palms.
Your blind appeal to authority would hold more water if we haven’t already seen national adoption of AI products in emergency services that that have catastrophically failed to perform their basic functions, leading to real world harm. Audits of Flock cameras deployed in several cities found an error rate ranging from 33-70%. People are being arrested for things they didn’t do because the AI doesn’t work and nobody cares. People are being stalked because nobody thought of basic security controls.
We’re talking about an AI implementation, not some rigid set of if...elsestatements.
Even worse, we’re talking about an AI implementation of if…else statements. Regular if else statements are consistent. As soon as you throw AI into the mix they become unpredictable. You introduce a failure mode that didn’t exist before that can and will have real consequences.
I believe the argument is that the probability of misclassifying an arbitrary segment of speech as the very particular answer “yes” is sufficiently low (with reasonably trained, current models) that we may ignore it under practical circumstances.
I find the non-zero probability of misclassifying in a little disquieting, but I suppose one way of looking at it is that this will probably increase the expected value of lives saved; the very probable case (calls are correctly filtered and a smaller proportion is passed through, allowing operators to respond to more new emergencies) may save a lot of lives, whereas the very improbable case (something that is not “yes” is misclassified as such) may endanger a few.
One thing that would worry me about just looking at expected values is the possibility of bias against a particular group of people or emergency type, but to me that seems unlikely in this case.
Edit: it also just occurred to me that if you are woefully unfortunate, you can probably just call again if you accept the assumption that there is a high probability the answer is yes given the model classified it as such. It might be more or less random chance?
Yeah I just don’t buy that there is a case where any “net gain” here is justified. These are people’s lives that you’re playing with by reducing them to what amounts to a balance sheet in the name of saving money.
The idea that you would create the possibility of denying someone emergency care that didn’t exist before, no matter how improbable (probably not even that improbable, given the propensity even the most advanced frontier AI models have for getting things wrong or hallucinating entirely despite simple instructions) just to save 10-15 seconds on average for other callers is not only absurd on its face, but morally bankrupt. You can personally ignore it because you don’t live there and it’s not the lives of you or your family at stake. I think those who this system could fail would not be able to ignore a flaw like that when it happens to them, and the idea that their peril was “highly improbable” will not be of much comfort to them.
The problem is that there are staffing shortages. The solution is hiring more staff. Trying to cheat our way out by implementing a system prone to unmitigatable flaws that could have life-altering or even life ending consequences isn’t a solution, it’s dystopian.
None of that applies if a caller just can’t get through at all because the bot mistook “no” for “yes”
Do you honestly think that nobody has thought about that and solved that problem already? Even with all the engineers involved, and after all the real-world testing, you’re the first to have considered that scenario?
Even consumer-grade products like ChatGPT can be interrupted while talking. We’re talking about an AI implementation, not some rigid set of
if...elsestatements.That is the thought process of a child. If your defense of something with obvious issues is “Well I’m sure they know what they’re doing, because if they didn’t it would be bad”, I have some really, really bad news for you about the world.
If this was thought through by a capable adult, it would’t be implemented at all. The very fact that this solution is on the table is proof of the breakdown of rational thought. Or more likely, greased palms.
Your blind appeal to authority would hold more water if we haven’t already seen national adoption of AI products in emergency services that that have catastrophically failed to perform their basic functions, leading to real world harm. Audits of Flock cameras deployed in several cities found an error rate ranging from 33-70%. People are being arrested for things they didn’t do because the AI doesn’t work and nobody cares. People are being stalked because nobody thought of basic security controls.
Even worse, we’re talking about an AI implementation of if…else statements. Regular if else statements are consistent. As soon as you throw AI into the mix they become unpredictable. You introduce a failure mode that didn’t exist before that can and will have real consequences.
I believe the argument is that the probability of misclassifying an arbitrary segment of speech as the very particular answer “yes” is sufficiently low (with reasonably trained, current models) that we may ignore it under practical circumstances.
I find the non-zero probability of misclassifying in a little disquieting, but I suppose one way of looking at it is that this will probably increase the expected value of lives saved; the very probable case (calls are correctly filtered and a smaller proportion is passed through, allowing operators to respond to more new emergencies) may save a lot of lives, whereas the very improbable case (something that is not “yes” is misclassified as such) may endanger a few.
One thing that would worry me about just looking at expected values is the possibility of bias against a particular group of people or emergency type, but to me that seems unlikely in this case.
Edit: it also just occurred to me that if you are woefully unfortunate, you can probably just call again if you accept the assumption that there is a high probability the answer is yes given the model classified it as such. It might be more or less random chance?
Yeah I just don’t buy that there is a case where any “net gain” here is justified. These are people’s lives that you’re playing with by reducing them to what amounts to a balance sheet in the name of saving money.
The idea that you would create the possibility of denying someone emergency care that didn’t exist before, no matter how improbable (probably not even that improbable, given the propensity even the most advanced frontier AI models have for getting things wrong or hallucinating entirely despite simple instructions) just to save 10-15 seconds on average for other callers is not only absurd on its face, but morally bankrupt. You can personally ignore it because you don’t live there and it’s not the lives of you or your family at stake. I think those who this system could fail would not be able to ignore a flaw like that when it happens to them, and the idea that their peril was “highly improbable” will not be of much comfort to them.
The problem is that there are staffing shortages. The solution is hiring more staff. Trying to cheat our way out by implementing a system prone to unmitigatable flaws that could have life-altering or even life ending consequences isn’t a solution, it’s dystopian.