Hi HN. I'm Jonathan, one of the founders. I'll monitor this thread all day to answer questions. Thanks for taking the time to read about us! It means a lot to everyone here at GetScale.
Do you think your camera system could (eventually) support some combination of Computer Vision and Machine Learning to identify (some) assembly mistakes in real-time and provide feedback/warnings to the assembler?
My thought is improving yields by reducing defect rates caused by those hopelessly imperfect humans (like me) might be a great selling point for some types of manufacturing.
You are describing a type of machine called an AOI (automated optical inspection). It is a common piece of equipment in electronics assembly lines. But AOI equipment is very expensive (~$250k+) and only works on a specific set of products. For example, it doesn't work well when the product contains soft or flexible surfaces, or has a highly non-planar shape.
GetScale terminals can be interfaced with AOI machines in the factory to collect and store the findings along with our human-led inspections and catalog everything by serial number and we do have ambitions to add AOI-like capabilities to our hardware down the line.
Sounds like you have a project in mind. What are you working on? ;-)
I was trying to avoid running off into the endless weeds of details and
acronyms and talking, but oh well... ;)
I'm not very familiar with the more advance microscopy based AOI systems
used on silicon and chip level manufacturing (e.g. wire bonding checks).
I'm also not familiar with advanced AOI used in robotic assembly. On the
bright side, I am familiar with the comparatively "simple" form of AOI
used on PCB assembly lines.
With PCB's most of the AOI identification and spatial orientation issues
are solved through Fiducial Marks [1] and pattern recognition. Some PCB
AOI systems even read/record chip/board lettering with OCR, as well as
handling Bar and QR codes. Most of the rules and routines controlling
the AOI are consistently derived directly from the PCB design files, so
there is little need for custom setup programming on each new PCB run.
That last point is critical; The rules and routines are automatically
and consistently _derived_ from the authoritative data sources. Manually
writing every tiny step in the AOI automation routines for each PCB
design would never be time/cost effective, so it has never been done.
Given sufficient example 2D images, 3D scan models, or CAD files, doing
the object recognition and spatial orientation with OpenCV is reasonably
straight forward, even without fiducials. The extremely tricky part will
be automatically deriving rules and routines for the GetScale AOI-like
capabilities from authoritative data sources. If the human assembly task
is to attach the top cover and secure it with four screws from the other
side, you should _never_ need to manually write the AOI routine for
checking the existence of a screw in each of the four corners when
viewing the bottom of the product.
I'm sure you're really busy, so I think I'll stop here. ;)
If you want to know about the projects I'm working on, email would be a
lot better for me. My address is on my HN profile page. Also, I spotted
a few places where your website could use a little TLC, and it's better
to point them out through email rather than publicly.
We've discussed this possibility of this and I would really like to make an attempt when we have more data!
The practical application is this is sort of an 80/20 proportion though. Currently we see fairly significant improvements in some situations just by the system being there. Factories will pre-test units before passing them over to our system in order to demonstrate better quality, when prior to installation they weren't testing a majority of units at all.
There are metric-truck-loads of existing research on behavior changes
due to known surveillance. Though they might be pre-testing units
off-camera for the sake of giving a good "demonstrating quality" stage
show while being recorded by your system, there's also the effect of
better behavior when knowingly being monitored. Either way, you win. ;)
The problem with pre-test is it can both decrease defect rate _and_
decrease throughput rate. Depending on how much each decrease is, you
can actually decrease yield/time rather than increase it. This exact
problem exists on PCB manufacturing. It's certainly possible to pre-test
every component as it comes off the reel and before it's used on the
PCB. The component pre-test does decrease defect rates of the finished
boards, but it also decreases throughput rate, so it can negatively
impact yield/time. The decision then becomes an investment/accounting
problem. Often, post-mfg circuit/component test automation and reworking
failed boards makes more sense financially.
As far as I know, no one does hard or soft real-time verification of the
human side of manufacturing and lab experimentation, but _my_ knowledge
of the current state of the art is admittedly outdated. Using CV and ML
to spot human manufacturing errors (or inefficiencies) seems feasible if
you throw enough compute power at it, but whether or not it's practical
and financially viable is another (more important) question altogether.
Though CV+ML may not be reasonable in the real world, I agree with you;
it sure does sound like fun to try.
My thought is improving yields by reducing defect rates caused by those hopelessly imperfect humans (like me) might be a great selling point for some types of manufacturing.