[burgeralarm]

This is way cool, though I can't help but think that the results might differ from actual A/B testing because the testing context is explicit. Have you done comparisons to backend A/B testing to see how well the two align?

It would seem that traditional A/B testing allows you to see what actually converts, while this framework would be biased towards user preference--which doesn't necessarily imply conversion. For example, I think Amazon's site is ugly and busy, and given the choice between that layout and a cleaner one, I'd probably choose the cleaner. That said, there's no way they haven't tested the hell out of the home page and discovered that a busy page, though uglier, converts better.

[jiaaro]

That's true and that's part of the reason I built it. It's not intended to be a replacement for A/B testing, but a compliment.

A/B testing is great for lots of things, but it's not always the right tool for the job.

I use the term A/B testing loosely since it's obviously not actual a/b testing. It's a hybrid of a/b testing and a survey that allows you to ask a question an get much higher resolution, and test many more variation than a typical survey question by using some techniques from A/B testing