[vidanay]

I have worked in industrial machine vision for the last 18 years, and this summary (I didn't read the paper) reflects my comments to management every time they bring up the topic of AI or ML. Our inspection systems operate at anywhere from 300 parts per minute to 3000 parts per minute. AI/ML has way too high of a false reject rate (or even worse, a false accept rate!) The worst scenario I explain to my managers is if we implement a AI/ML system and for some reason it starts rejecting 50% of the customers product at 3am on a Sunday, then there is no practical way to analyze the results to determine a difinitive cause for the reject, and what the corrective action needs to be. The final gut punch is then we would have to tell the customer that it could be several hours to retrain the model(and that's only after we figure out what needs to be represented in the good and bad image sets to account for the new failure mode).

[andrewtbham]

I recently spent months implementing ML CV system for manufacturing and I tried several of the services mentioned.

This paper is comparing AWS Rekognition, Google Cloud Vision, and Azure Computer Vision. I tried Google and Azure. I also tried clarifai.

To get 100% accuracy I ended up building my own service, and doing some really unconventional things. I used tensorflow, but I may rewrite the whole thing in pytorch.

I tried using neural architecture search but it was a dead end.

The key for me was training data distribution search.

https://twitter.com/karpathy/status/1175138379198914560

[PacifyFish]

Why not do a slow rollout, starting at maybe 5% of product. Then you can catch and fix deficiencies without compromising your entire production line, and ratchet up to 10, 15, ... 100% as you gain confidence in the model?

[vidanay]

Because no customer will pay for that scenario. That could potentially be a viable plan with an in-house implementation but not for a turn-key product with service agreements and performance guarantees.

[p1esk]

So what are your inspection systems based on?

[ohazi]

Vision systems in controlled environments like factories that need to be fast and stupidly reliable are often based on "classical" (non-ML) computer vision techniques.

The ML hype train has led to a frustrating amount of throwing the baby out with the bathwater, where people who should know better decide to use ML for more things than is reasonable (something something "end-to-end").

Ideally, ML should be used for a few very specific tasks, and then classical machine vision / geometric analysis / plain old logic should be doing the rest. If you don't do it this way, you eventually end up with the problems described in this paper, where performance is inconsistent and impossible to debug, and nobody can tell you what's going on or why.

[michaelbuckbee]

This is years ago and not my specialty, but I worked with a product line that manufactured what were essentially highly precise plastic washers. As they came off the line there was a clever bit of mechanical disbursement that slowly dropped them onto a conveyor under which a flashing light sync'd with a digital camera system took pictures.

Because all of the factors are known you could almost do pixel counts and compare to "true" circles (no need for AI models), any washers that didn't meet the critera were pneumatically puffed off of the line from an air hose.

[throwaway6734]

Thanks for sharing this!

Any tech that you think shows the most promise?

[vidanay]

Yes. AI/ML.

Like mentioned in another comment, targeted application of AI/ML can provide excellent results when the expectation is carefully defined and the objective is clearly stated.

Many AI/ML proponents jump directly to a 'black box' attitude where a product enters one side and leaves the other side and a smiley or frown face tells you if the product is good or bad. There are no definitions of dimensional tolerances nor geometric features. It's just good or bad based on the training model. But unfortunately, GIGO applies very strongly to the training model.