How accurate is the counting? For examples like the case study, I presume your customers will want very close to 100% accuracy, but many other use cases will only need rough numbers. How do you calibrate / verify your app?
You have to take into account the error rate of human counting when making the comparison.
I can also imagine the new process have other benefits. Like before, if a customer complained of getting fewer than ordered pearls, they would probably have to acknowledge that as a counting error on their side but not knowing. Now they have a picture/data to refer back to.
At least that's how it is where I work; the automation leads to reduced errors, and when they happen the amount of data makes it easier to track where it went wrong.
For most of our clients, we get to 100% accuracy. For some more difficult scenarios, a lower accuracy is ok for estimations (e.g. estimating the crop in agriculture).
It's also super easy to correct mistakes or to add items that are not visible in the image.
You don't mean 100% accuracy, that means mistakes are impossible. Perhaps your error rate is very rare, but it's a bit concerning you don't attempt to quantify it.
I disagree. If we consider a single customer with a particular application, we probably imagine an iterative process like this: the customer supplies some sample images, the system gets a few wrong, they add special cases, fine-tune the hyperparameters, whatever, and after all this they literally get 100% on the customer's holdout data. If that happens for several customers then the GP statement is justified (no other number is possible for these customers).
Possibly you're thinking of a single error rate across all customers? For other customers, as stated, it's not 100%. But taking an average across multiple customers is not meaningful when some are counting pearls and some are counting crops.
Suppose it depends on which part of the chain he attributes error, plausible lighting or poor positioning - anything that is an employee duty - does throw it off and then they scrub it.
But yeah. Still.
The article shows a photo with the pearls highlighted with circles. I'm generally skeptical of AI, but this system produces a certificate that a human can verify much faster than doing the work themselves. IMO, that's the right way to do it. You can trust the system 99% of the time, and check a random 1% by eye every day. Or, you can keep a human fully in the loop and still save time (though, if the error rate is too low, humans will get bored)
I can also imagine the new process have other benefits. Like before, if a customer complained of getting fewer than ordered pearls, they would probably have to acknowledge that as a counting error on their side but not knowing. Now they have a picture/data to refer back to.
At least that's how it is where I work; the automation leads to reduced errors, and when they happen the amount of data makes it easier to track where it went wrong.