[mdeck_]

> It sounds like you would need to be exceptionally careful that your meta-process didn't "find" some signal in pure noise (via re-using test sets and so on).

It sounds like you’re actually talking about ordinary levels of carefulness in this (ML) context.

[jkaptur]

That would be great. I'm no ML expert, but my impression was that standards varied widely from team to team.