Hacker News new | ask | show | jobs
by makewavesnotwar 2739 days ago
It's incredible. The last company I worked for before going it alone (I was the front-end engineer and moved away from ML based business models) was trying to automate statistical analysis. I came from an academic background in economics and I tried to propose modelling to them in simple terms and they jumped to, "it sounds like you're talking about forecasting, let's go with that." And they started trying to implement python models from academic papers with highly limited training data and my reaction was generally, WTF?! You can't even start to forecast without a reliable base model. But they went ahead trying to sell stuff like churn prediction to companies with 0 understanding of how these models work at the basest levels.

And yeah, Google started to throw their hat in the game with Analytics 360 and an enormously larger training base. Amazon's another major player.

Weirdly enough though, people do still blindly pay my previous employers to figure stuff out because easy answers are always actionable, even if they're wrong. It's just crazy because the CEO explained to me that lying about the service to potential customers and investors was necessary because "Faking it til you make it" was a sound business principle in his mind like 1980's Michael J Fox was his primary sources of business info.

Long story short, don't waste your time with these little companies purporting ML holy grails. They're probably just lying to you, whether intentionally or not. ML is a game for the big boys with access to market level aggregates. The models that last company came up with were wildly inaccurate.

4 comments

I only partially agree. Building good ML models and even outperforming the ML services of the big players is absolutely feasible. Have e.g. a look at this talk from PyCon DE (in English: https://www.youtube.com/watch?v=XniwzOCWi2c), which shows how a small team built a machine vision system to read car registration numbers from official documents. The system was built and trained with an extremely small dataset (I think around 60 scanned documents with some data augmentation) and was able to easily beat the Google Cloud ML algorithm by an impressive margin (Google ML had an intolerably high error rate for this seemingly simple problem).

So I'd say if you have a very specific area that you're investigating you have a very good chance of beating larger players that don't specialize as much as you can. Of course competing against Google in self-driving cars or machine translation might be a bad idea, but even in those areas there are small startups that produce impressive results (e.g. DeepL: https://www.deepl.com/en/translator). Also, big companies regularly exaggerate their capabilities as well (sometimes more than startups), just have a look at how IBM markets their Watson AI/ML solutions, and what they deliver in reality.

So personally I'd say it has never been that easy to build relevant and interesting ML/AI based solutions as a small team, and it is possible to beat large players if you have the right approach and the right (very narrow) problem.

DeepL is a very promising thing. I was very sceptic on the future of automatic translation seeing as Google Translate seems to have stagnated for the last two years or so, but I’ve just recently tried DeepL on a German newspaper article a couple of days ago and it did a very good job. Granted, I don’t know German (hence why I used DeepL) but nevertheless the English translation provided by DeepL seemed more polished than what Google Translate usually does.
I've used it a fair amount, and continue to be amazed with the quality it puts out. There are still some issues with formal pronouns, subject-matter-specific contractions etc, but otherwise it does a great job with both EN->DE and DE->EN
Oh yes. I've seen this plenty of times not with just ML, but even basic statistics (and by seeing I mean working next to people doing it). You don't need to understand statistics at all, as long as your customers don't understand it either, and you sound confident enough. If it's hard to verify whether a model works, you can keep the customer happy and yourself paid while not providing much, if any, value to them.

I currently believe this is how most ad tech runs internally. Scammers scamming scammers.

That's obvious to anyone with any sort of basic idea as to how machine learning works. Feed bots data and test the bots predictions - the more data you have then obviously it'll be better. If you have 1 picture of a bee to test you'd only ever get one very specific shape for a bee from any AI, if you have thousands, you get a much better representation. It's pretty simple.
This is mostly true at the minute, but not completely and probably not for the long term (10 years +) See http://science.sciencemag.org/content/350/6266/1332.full

Also you can have problems where more and more data doesn't make an impact - the stock market being a clear example.

It might have been the case before. Turns out you don't need that much data today with intrinsic reward functions.