[ska]

    For years we used only algorithms based on basic statistics but we've found places where the machine learned models out perform the simpler models.

This is the right way to approach it. Too many people are looking for "deep" as some sort of silver bullet for an ill defined problem they have. If you can't validate against a simple model trained properly you are already in trouble. Likewise if you don't understand how to evaluate your generalization issues and how/if a simpler model will improve them.