"If all you have is a hammer, everything looks like a nail"
The model I've consistently chosen (Decision Trees) may not be the best model. Need to get pushed outside my comfort zone.
I could put in the months/years like a proper Data Scientist and optimize the model. Or let a magic API tell me the best model. I'm lazy, so I prefer the latter ;-)
Weka and various ML tools require you select the algorithm and do the A/B testing on your own.
There's an opportunity for an Optimizely of ML.