|
|
|
|
|
|
by Mister_Snuggles
2435 days ago
|
|
|
> I’ve always had a hard time understanding the value proposition in the same way I don’t understand the value proposition of e.g. AWS Rekognition. I signed up for AWS specifically to use Rekognition. I use it to screen alerts from my security cameras. In short, Blue Iris detects motion, a Node-RED flow grabs an image and uses Rekognition to see what's in it, if there's a person detected the Node-RED flow notifies me via PushOver. This significantly reduces the false-positives that inevitably happen on windy days - I've already done a lot of work in Blue Iris on this, but passing alerts through Rekognition makes it almost perfect. Based on my testing, this reduces false-positives to zero and hasn't yet produced a false-negative. Based on my usage I expect my costs to be ~$5/mo once I'm no longer in the free tier. This is cheaper than the person detection service that Blue Iris natively integrates with and is significantly less effort to get up and running compared to, for example, TensorFlow. I also assume that Amazon will periodically update their detection models to make it better, which is one less thing for me to worry about. For me, all of these benefits are worth the ~$5/mo. |
|
|
But you’re just proving my point. It wouldn’t make sense to use Rekognition unless you had someone with skills to assess the classifier accuracy in the context of your specific problem. For example, it seems like your loss function places an asymmetrically higher cost on false negatives. (Incidentally, it’s interesting you claim it hasn’t produced a false negative ... did you watch every frame of video and make sure?)
If you replace your simple one man operation with a simple loss function on an amount of data you can manually evaluate with instead a complex computer vision workflow, say where face or person detection has legal consequences for a company that sells or licenses stock photography, or an image or video search tool trying to avoid surfacing porn or pirated content, etc. then Rekognition becomes no longer useful, because you’ll need not just one person doing cursory evaluation of false negatives, but a team of people building out a benchmark-like battery of automated evaluations with probably IoU metrics in addition to classifier metrics and will need to figure out how many errors they can tolerate in some cost budget combined with the normal cost budget of usage to Rekognition.
Basically, for some tiny hobbyist use case, I guess it’s fine (though really you could literally just load some Keras model pre-trained on imagenet or some off the shelf version of yolo and save yourself $5/mo) but the value proposition falls apart as soon as the cost function becomes a complicated business one.