[autokad]

> "VMs would be a terrible idea because the hinge loss only focuses on two data points and essentially ignores all the rest"

Not true, In my experience fitted SVMs have thousands of support vectors. The hinge loss is supposed to be less sensitive to outliers.

In general, I think SVMs are a 'terrible idea' because you can often get better fits at much faster run times with gradient boosting or you have to spend a lot of time getting the kernel just right.

[uoaei]

I'm eschewing kernels entirely and just sticking to linear models for reasons around interpretability (need to convert the model's coefficients+intercept to an explicit Boolean statement). But you're right that kernel methods are more flexible (maybe too flexible).