[slackdog]

You should design your site so that you can troubleshoot it without using real user data. Failing that, you should receive voluntary informed consent from the users, and exclude those that don't consent.

A/B testing / etc is a form of human subject research, the only way to experiment on humans ethically is to have voluntary informed consent. The fact that you're 'merely' running psychological tests on website users instead of performing medical vivisections doesn't free you from the moral and ethical obligation to acquire voluntary informed consent.