[kqr]

The thing you seem to be missing is that there's no one number that's a meaningful limit for all purposes.

What probability you accept as significant should depend entirely on how you plan to use the results. Something with a p value of a staggering 70 % (i.e. it's more likely not true than true) is significant if the payoff is good when it's true, and the cost is small when it's not true.

And 70 % is very far from 5 %!

Then again, if the payoff is tiny compared to the cost, you might ask for a p-value of less than 0.01 %, in order for it to make sense to take the chance on it.

Think like a poker player: a hand that has 1/4 chance of winning needs better than a 3-to-1 payout when it wins to be playable. Conversely, when the pot offers you a 3-to-1 payout, you better make sure your hand has more than a 1/4 chance of winning.

[robotresearcher]

I understand and agree with that. Pascal’s Wager is the extreme example: unlimited upside makes the likelihood irrelevant.