[mykowebhn]

I worked at a well-known clinical psychology lab at an Ivy League school many years ago. There was so much "gaming" of the results.

For example, our PI didn't want to include a subject in our study because his scores weren't elevated enough, and our PI was worried that his score wouldn't drop enough which would adversely impact our results.

Another example: our active treatment therapists knew exactly what they were treating for, and our study was measuring improvements in the condition that was being treated. However, the control therapists had no idea what they were treating for, and we purposefully kept this information from them!

[kizer]

Hence the “replication crisis”. Replication of experiment and differences in outcome highlight the “core” result. When the results are just totally different, obviously there was “gaming” or just a bad experiment.

[mattkrause]

The first example doesn't seem that bad. No intervention cannot reduce something if it isn't there to begin with.

The screening should be disclosed in the methods (and, ideally, pre-specified), but you do need to account for floor/ceiling effects somehow.