[boyakasha]

The central point of the article is that certain people (designers, for lack of a precise term) can use their intuition to get better results than actually testing alternatives; to quote the article:

"Testing can only tell you so much -- and it often only reveals that people only like things that are similar to what they've had before. But brilliant design solutions convert people over time, because they're both subtle and ground breaking."

The article provides no evidence to support its claim. For all the anecdotes, not a single one describes a situation where the intuitions of "designers" outperformed a solution based on testing alternatives.

However, the pro-designer propaganda flies thick and fast, repeatedly implying designers would have made a better solution, or seen the solution as obvious without the need for testing:

- "While that solution seems obvious and not particularly elegant"

- "Is it just us who find our eyeballs spinning in their sockets...?"

- "Obviously, none of these (prototypes) were going to work"

- "A little design know-how would have made that obvious"

- "(They eventually settled on a blue that is basically the average of all the blues used in hyperlinks across the web. Duh.)"

- "Google's 'solution' to providing instant results still seems so primitive and ugly"

- "But brilliant design solutions convert people over time, because they're both subtle and ground breaking."

- "testing artificially limits the worldview of the people"

- "has your G-mail or Google Reader gotten any easier to use, or less stressful on your eyes? Have either of them become a pleasure to look at or play with? No."

The author isn't interested in making a coherent argument. The author's interest lies in unashamedly gratifying the readers' sense of importance. It is a website for designers, after all.

[mattmanser]

I felt that the central point the article was trying to make, and I thought made it well, is that design is not about testing a set of obvious solutions and then declaring X the winner because it was tested well.

Firstly that often results in horrid UX experiences, as Google Instant is. It is also, weak for the following reasons:

1. No innovation happens (which is why Apple came up with the smartphone touch UI and Google just copies it)

2. People don't like change, so will favour things close to what they had before

You have fallen straight into the same odd thinking dominating Google, that UI is incremental, testable and easily measurable.

You want precise and measurable. That's not design. Jonathan Ives is worth his weight in gold for his design skills as much as Linus Torvalds is worth his weight in gold as a programmer. And we as engineers have to accept that.

Google will never come up with a great design with the way it approaches the problem. While I love their products, none of them have ever blown me away because of their elegance or coherence. They only know how to do simplicity and are beginning to forget how to do that too.

[boyakasha]

If Jonathan Ives submitted a design for Google Instant, and his design was better than the "obvious solutions", then it would test well. If a better solution tests badly then, by definition of the word "test", you're not testing it properly. Whether or not the test candidates are "obvious solutions" or created by designers has no impact on whether testing is good or bad. Maybe someone can test whether testing tests better than not testing :)

I'm not saying that designers' intuition can be replaced by testing, I'm only saying that testing can not be replaced by designers intuition as the article implies. A designers' intuition, after all, hopefully comes from the experiences of informally testing out designs in real life.

What really annoyed me about the article was the attitude of "We designers come up with brilliantly ground-breaking yet subtle designs, and if testing says there is a problem, then ignore the tests because we are always right. Isn't that right, boys? YEAH! WE RULE!"

[KuraFire]

"If Jonathan Ives submitted a design for Google Instant, and his design was better than the "obvious solutions", then it would test well. If a better solution tests badly then, by definition of the word "test", you're not testing it properly."

Innovative products rarely test well, overall. If you test it to a group of visionaries who are comfortable thinking outside the box, then yes, it'll test well. But if you test with a group of regular / casual computer users, non-techies and the like, many innovative or important design decisions may not test well.

Take, for example, USB. The technology came out and was sparsely added to new computers, but always in addition to COM and Parallel ports. Apple saw that USB was the future, so they made the iMac which—gasp!—only contained USB ports. No COM, no Parallel, just USB.

You think that decision would have tested well with users?

"I can't use my existing printer?! That's stupid!"

Except… it was this very decision that made USB successful in the market, because it forced all peripheral manufacturers to make USB devices if they wanted to sell to Mac customers (a large enough market to be worthwhile for virtually all of them).

And so, USB succeeded in the market. But it wouldn't have if Apple had used Google's test-driven approach, because that decision would have tested quite poorly.

[tyen]

Hang on, Apple also did the same with firewire and it died even though it was a technically better solution. Its error was it was expensive to manufacture a firewire device as it needed a hardware layer on the device side. USB didn't, so USB devices were cheaper and they won the war once USB 2 came out and the speed difference was negligible.

So not all Apple decisions work, like PC's over Macs. Apple lost that war as the open standard won.

And then sometimes it is the better design, like iPods and iPhones over their competition that wins.

My point is I'm not sure your argument holds in general, sometimes its this, sometimes its the other thing that works.

[mattmanser]

I still feel you're mischaracterizing the point. Perhaps it might be seen as this, using testing you start from a position, create a few obvious alternative, test, iterate, resulting in a > b3 > c2 > d6 > e3 > f4. You then declare f4 is amazing because it beat the gradual iterations that got it there.

But a designer's there to sit down and go, right, what about z. And j. And 77883. And still not test them, but decide on one. And then refine it and then you test it some and refine it some more.

And even better is that the designed product will have a flow, a coherence because it's not about a bunch of tiny improvements and changes, it's about a vision.

It wouldn't necessarily test well against a to begin with. People don't like change, so say here's Ives' search and here's the instant that's almost exactly like the existing google and you've got a lot of initial resistance that will skew the testing.

But people who love new stuff and then evangelize, mavens I think Gladwell called them, will result in more people trying it and it will end up successful. The people who helped push Twitter and Facebook and Google itself. Not that I buy all of tipping point, but there are some good points in it.

Incremental design is not about design at all, it's more about fear. Designing can include A/B testing, but not at the start. It just seems the wrong way to go to me.

As for the tone, I didn't notice it, but then again I realized after reading it I already agreed with it, he was vocalizing something that has been dawning on me.

[boyakasha]

I think we have different interpretations of "testing". I'm talking about testing in general, not just testing incremental changes.

[points]

"Firstly that often results in horrid UX experiences, as Google Instant is"

Am I the only one that loves the UI for Google Instant and thinks it's the single biggest time saver of recent times?

[aik]

I completely agree with you. Today I found myself using a search in another tool and sat there for a moment after I typed, waiting for the results to come in. Once I realized the results weren't going to come until I press enter, I found myself a very tiny bit frustrated that I had to exert additional effort for me to see results.

[vessenes]

Nope, I love it. Makes it faster, so it makes it better. (And I'm a fast typist already).

[brown9-2]

1. No innovation happens (which is why Apple came up with the smartphone touch UI and Google just copies it)

How do you know that Apple's designers aren't also testing their designs with real world users?

I don't think "design vs real world testing of experiments" is a binary choice.

[KuraFire]

"How do you know that Apple's designers aren't also testing their designs with real world users?"

Having worked there, I can vouch for this. Apple employees (and, most notably, Steve himself) are the real world users they test with. Every important or critical product only ever sees internal testing, but much more important is that the rationale behind Apple's design process is that of an actual designer thinking about solving problems from a user’s own perspective, whereas Google's "design process" is based entirely on trying to solve problems from an engineering perspective.* Often, they take it so far that they only solve engineers' problems, not even general users' problems.

* That's bad because the vast majority of people on this world are not dedicated engineers.

[mattmanser]

Neither do I, I think anybody would be crazy to not test their design. I'm just not confident in design as a primary design mechanic.

[lhnz]

I don't know. I think Google Chrome has the best default UX of any browser I've used. And I wouldn't say anything about that was incremental...

[GiraffeNecktie]

Nothing about Chrome is incremental? Really? I would have said the UX was entirely incremental. Yes it was arguably better than any other browser at the time it was released but putting the same stuff in different positions or removing some extraneous elements is, to my mind, an incremental change.

[mattmanser]

And doing what Google does best, it was simple.

[duck]

And doing what Google does best, making it faster (and I think that is why most of us use Chrome).

[njharman]

> is that design is not about testing a set of obvious solutions and then declaring X el ganador(the winner) because it was tested well.

If that is true then Google's immense success proves "Design" is not the only way to design and strongly points towards big D Design with traditional designers being not the best way to design.