[hkuo]

I think the first part of "looking for a designer" is the hardest and most important part. As with any profession, not all designers are created equal. I would look for a designer who emphasizes user experience, not just someone who makes pretty pictures. I would make that the first topic in any discussion with a potential designer. If they have no experience or regards to UX, then keep looking.

[karamazov]

Once you meet some designers who do have experience with UI, how do you evaluate them? It's not as hard as evaluating a programmer with no knowledge of coding - design is much less of a black box, and it's at least possible to tell when a design is bad - but how do you tell a good designer from a mediocre one, and a great designer from a good one?

[hkuo]

I don't think there are any hard and fast rules, but I think a simple question that will give you an immediate sense on whether a designer knows their sh*t (pardon my French) is to ask them what their design process is. If there's any hesitation, or if there's a pause and you can tell they're trying to make something up as they go along, they don't have one. Move on. But just pay attention to what they're saying and you'll likely be able to tell how experienced they are and how confident they are of their skills.

Edited to add: this is where you'll find out if UX is part of their process. If they begin with things like mood boards, font selections, color palettes, look & feel, no, no and no. It needs to begin with some form of audience/customer, business goals, content strategy, user flows, functionality. Those other things come after, not first.