[dasboth]

Depends what you're tuning. If it's something like the number of trees in a random forest, definitely do that automatically. If it's the number of clusters in a clustering problem, that's where you'd be asking an expert something like "how many distinct groups of customers do you think we should try to split them into?" and go from there. But even in that scenario, the expert's opinion might just be your starting point for automatic tuning.