[imurray]

One should often consider applying tweaks to a final system. If there's an obvious place to introduce a free parameter, it seems silly not to do so and cross-validate the parameter against application performance.

Things get out of hand if there are many such possible tweaks, or multiple components are combined, each with interacting tweaks. Then some principles behind the tweaks need identifying. Or at least a differentiable cost function to target.