I'm sorry, but after seeing numerous self driving cars plow into people and no clear testing about what's safe when the platform already provides instant access to a live human, I'm really worried about technologies like this.
That death is of course tragic, but it's important to put it in context.
That's the only death of a self-driving car. That's the only one. There was a safety driver with their hands on the wheel, and they didn't see the pedestrian either. And that was an Uber self-driving car, and they've since canceled that project.
While incredibly tragic, it's not at all fair to say that self-driving cars are constantly plowing into people.
Waymo has driven more than 1 million miles with no human injuries. That's dramatically better than any human driver.
Cruise is a close second in number of miles, also with no human injuries.
Be My Eyes has existed since 2015. If trolling is common, there should be reports of it online. A cursory search hasn’t revealed anything, but maybe you’ll find something.
Trolling isn't a big problem, but many humans have good intentions but just aren't that helpful. Some want to be chatty or give unsolicited advice. Some just aren't very good at tech. Some just aren't very good at describing things clearly.
Is this some sort of sarcastic joke? No, I absolutely don't think that people who are unable to see could know whether the thing they cannot see could accurately be described by an AI.
Can you give an example of the dangerous situation you are imagining here? I do not see how this app could be dangerous unless combined with poor judgment and failure to follow instructions (e.g., do not use to read medicine labels, do not use to cross the road).