[DeathArrow]

I believe neural networks are over hyped sometimes.

They are not always the best tool for the job. There are lots of other ML techniques such as SVM, naive Bayes, k-nearest neighbor, decision tree, logistic regression, random forest etc. nobody is using because they lack the hype factor.

If something lacks some keywords like neural network, deep learning, reinforced learning, than it is deemed not cool.

[learndeeply]

I can't think of anything that neural nets can't beat, except small tabular data with boosted decision trees. Can you give some examples?

[zelphirkalt]

Explicability is a big part of it It is often worth being a percent less accurat but having an explainable result.

[adamsmith143]

I've been on a lot of ML teams and outside of Finance and a few other sensitive topics explainability has always been irrelevant.

[zelphirkalt]

What happens, when your model exhibits a discriminating bias? How do you find out, what is going wrong? Knowing, what the model pays attention to can be pretty helpful.

[adamsmith143]

Not aware of any court cases where someone successfully sued because they were shown one product recommendation or Ad on a webpage instead of another.

[jstx1]

(I don't really agree with GP's point but for the sake of answering your question)

1. Collaborative filtering based on a sparse dataset of implicit interactions.

2. Many time series applications.

[marcyb5st]

Didn't all recommendations engines move to two-towers like models? I remember that it "solved" the freshness problem (ie when adding a new item to your catalog how do you recommend it to users if there are no ratings/interactions). Of course as long as you have a good model that creates items embeddings.

Regarding time series, don't everyone moved to attention based models?

Not challenging your answer, just curious. I work mostly with Graph NNs and quite a bit out of touch with the rest of the field.

[niemandhier]

Small data problems, where’re never the less have a really good idea of how things are causally related.

[insane_dreamer]

> we often use ML over DL in scientific analysis because we need models that can be inspected/explained not just results

> also, DL generally requires more data whereas you can get by with ML on less data if you have domain knowledge

[patrick451]

The black box nature of a neural net is a problem. For model based design, a bit more accuracy out of a black box doesn't really help when you need, for example, state space matrices in a control design.

[jack_pp]

I'm no expert but can you show how those techniques can be used to solve the same problems NNs can? Like SOTA image recognition, chess / go, STT, TTS etc?

[DeathArrow]

>I'm no expert but can you show how those techniques can be used to solve the same problems NNs can?

Sentiment analysis, classification.

[nl]

NN based sentiment analysis is certainly a lot better than non-NN based techniques.

Classification depends on the problem (and mostly the datasize). Boosting is certainly competitive on tabular data and widely everywhere I've worked.

No one talks about it (except on Kaggle) because it's pretty much at a local maximum. All the improvement comes from manual feature engineering.

But modern techniques using NNs on tabular data are are competitive with boosting and do away with a lot of the feature engineering. That's a really interesting development.

[Vetch]

> NN based sentiment analysis is certainly a lot better than non-NN based techniques.

I wouldn't say this. Sentiment analysis trained on the standard datasets is one place where performance is barely better than old-school linear classifiers. They remained brittle and easy to trick until recent flexible systems systems based on question answering, zero-shot entailment or lotsa instruction finetuning (improving in that order). I strongly advice against using something fine-tuned solely on sentiment datasets. It'd be a total waste.

[nl]

> Sentiment analysis trained on the standard datasets is one place where performance is barely better than old-school linear classifiers

Well yeah. But why would you do that?

Do what eveyrone does: Train on large scale a language corpus (or use a pre-trained model) then finetune for sentiment analysis.

> I strongly advice against using something fine-tuned solely on sentiment datasets

Did you mean trained on sentiment datasets? I agree with that.

Otherwise, well [1] is a decent overview of the field. I think Document Vectors using Cosine Similarity[2] at 17 is the highest rated that isn't a NN trained on large corpus and fine-tune on sentiment task. Even that uses document vectors that are trained on a large language corpus.

[1] https://paperswithcode.com/sota/sentiment-analysis-on-imdb

[2] https://paperswithcode.com/paper/the-document-vectors-using-...

[Vetch]

No, I meant finetuned. I also meant finetuned when I said trained. Experience with applying finetuned sentiment classifiers on real world data found gain vs cost of running to not be worth it. They remain nearly as brittle as cheaper classifiers and have a habit of gloming too much unto certain adjectives. They are also prone to overfitting on finetuned data's domain. Transformers trained not specifically on sentiment but on general domains like question answering or entailment are just leagues better for sentiment tasks.

[minimaxir]

The problems where traditional ML works best and the problems where Transformers or ConvNets work best are usually two different domains.

AI is not a buzzword.

[DeathArrow]

>The problems where traditional ML works best and the problems where Transformers or ConvNets work best are usually two different domains.

Yes and we are using NN for everything.

[adamsmith143]

But we aren't. Outside of using AEs for embeddings and then feeding them through a boosted tree model I don't know anyone using NNs for tabular data. We all use XGBoost or Catboost, etc.

[adamsmith143]

Don't think you really know the field. On my team we almost exclusively use XGBoost or other boosted tree methods because it is typically the best model for tabular data. If we were working on CV or NLP that would be a different story and for that Neural Nets are by far the best models.

[HelloNurse]

This is a library for neural networks, and it should be compared to other neural networks solutions.

[jstx1]

Everything you're listing works mostly on tabular data, not on text or images which is where we have the most impressive ML applications right now.