I'd hand them a piece of paper and ask them to sketch out a profile page that's user friendly.
Then I'd ask them how they'd design the schema for that.
Then I'd ask them to sketch out the admin interface for managing users and how they'd implement that.
If they start asking questions about permissions and roles and such I'd be really impressed.
Essentially I'd act like a client/end user who kinda knows what they want to see if they can ask the right questions.
I'm not looking for a UX expert but more someone who thinks about the process and end result as well as a nice way to implement the code side.
I'd ask programming related questions as well but if (when) I hire I want someone who thinks about the end user experience while they are developing, I don't live in a world where a job comes with a 400 page spec and neither would they.
I take the view that I'd rather have someone who can think about how the end system will work than someone who knows ever single API for whatever language I'm hiring for, one of those you can find on google in 30 seconds the other not so much.
Full stack engineers are generalist engineers who are motivated by technical problems. Product engineers are motivated by building impactful tools. They learnt programming somewhere along the way as a necessity to show the world their ideas.
Product engineers generally pick mainstream languages and tools. They talk about users, strategy and product.
That's an interesting way to distinguish them, though I've seen people change from one to another. Motivations can change in a person.
I'm currently in the process of teaching one of the agency devs we hired on contract how to think with the end-user in mind. At the same time, I've tried to teach other how to reason through things from a purely technical perspective.
"Product engineers generally pick mainstream languages and tools." <-- going by that characterization, a "product engineer" will miss out some strategic advantages that comes from learning non-mainstream languages and tools. This is coming from the idea that, say, there are powerful abstractions and concepts that can be applied broadly, including UX.
For example: Promise Theory is a formal language for describing intentions and a powerful framework for creating cooperating agents in the context of uncertainty. It is typically applied to configuration management, infrastructure and microservices, but it can be applied broadly to human-to-human and human-to-machine interfaces -- literally, UX. For example, knowing that humans will typically assess trust in an iterative way, you can pick apart the entire user flow in terms of the perceived (implied or explicit) promises (user expectations) the product makes. It starts from the advertising (signaling), through user signup, through onboarding, and any useful impact that product makes. It extends through into support and infrastructure scaling, and back to the user signaling to other users that, hey, this is a useful too.
Another huge aspect that conception of the product engineer misses is strategic thinking. Strategic thinking is the art of making decisions in face of uncertainty. Engineers typically don't think strategically, usually thinking in terms of relative advantages instead of strategic advantages.
What differentiates a "product engineer" from an entrepreneur†? Lack of ambition?
Insofar as I fit all the criteria you're listing, those traits are all things that push me to want to work on a small, independent product where I can dictate the vision for it. If I'm willing to "learn programming somewhere along the way as a necessity" to express that vision, there's no reason I won't also decide to learn how to start and run a business "along the way, as a necessity."
To put it another way: in the games industry, the "product engineers" there are called "game designers." Game designers don't generally work as freelance consultants, or put themselves on the market as employees for companies like EA or Sony to snatch up. Instead, if you have the actual vision required to come up with a decent game, you'll make it—and sell it—yourself, either as a one-man show, or by starting your own small games company.
The only game designers working for the big games companies are the ones that grew into that role from whatever they got hired to do (usually either programming or art)—and, even then, only embraced their roles once the company decided to start letting them come up with products "from scratch" that they could basically run as their own little ventures, rather than just stewarding along some existing product of the company's.
---
† I say "entrepreneur", but not in the typical HN meaning of the term. Most "startup founders" I ever hear about seem to be either pure-engineering types who just think about cool tech all day, or pure-business types who think about what'll be profitable all day and follow trends. Entrepreneurialism seems to actually be much more common outside Silicon Valley in regular-old small businesses, where someone with a "product vision" like "feeding people really weird pizza" will get a loan and set up a pizza place to fulfill that vision.)
Product engineers are entrepreneurs in waiting who have yet to find product-market fit with an idea of their own. They've typically tried a few times, but haven't yet won at the startup roulette.
Product engineers and entrepreneurs get along so well because they're the same breed of people. Great entrepreneurs are often product engineers.
And what of those who are motivated by technical problems and by user needs and business strategy? Is it impossible to learn programming because of an interest in computer science or in the challenge of programming for its own sake, but learn product and strategy as a way to put those skills to impactful use?
I don't like this sort of pigeon-holing; most developers I know straddle this divide.
"Full-stack engineer" describes a person's technical knowledge: comfortable doing front-end or server work, maybe comfortable designing whole things from scratch.
"Product engineer" describes a person's motivations: primarily interested in writing code as a means to making a product, or improving a product.
Lots of engineers are primarily motivated by writing high quality code, or very performant code, or prefer to write tools (libraries/frameworks) over writing products using those tools.
You're begging the question. What does "engineering the product" mean? Software engineers are not civil engineers, there are not professional standards for what this means.
Well, nice of you to give me hope that some people appreciate and even seek what I think I'm good at. :) I am quite happy where I am now anyway. Wish you luck!
It depends on the company, but I'm a big fan of pairing with a developer on something they're working on for an hour or so. The dynamics of the interview completely changes from "I need to figure out hidden knowledge that the interviewer knows already" to "I need to work with this person to find out a solution together", which I think is quite a bit less stressful. You're also working on real problems that — though they might not be a glamorous interview question — will be much closer to what "real" work at the company will be.
Put another way: I've probably done a dozen or two product interviews this year, and have still never seen a full stack web request. I mean, I haven't seen a controller action, or a view, or even a model for that matter. And that's the area that's my entire job, really: I build screens and hook things together and make things work for users. As much as feasible, I'd like to be able to demonstrate those types of talents in the interview process.
In general, anything that moves the interview from adversarial to collaborative is a big win. In many ways, interviewing is a lot less about assessing skill and a lot more about creating affection between two strangers.
That fits -- it's difficult enough to recruit people, let alone people that you can work with, or building out a gelled team that can work through on a lot of challenging things (and not all those challenges are technical).
I had one of these "pair" things by a startup touting how different their process was. It consisted of two developers doing work while I solved a thinly-veiled CS101 interview question alone, on a setup I wasn't familiar with, in an editor I couldn't change. I tried to ask questions but the typical answer is "we really cant help you". I'll take the whiteboard, thanks.
Completely agree with pairing. If possible, with more than one developer, to avoid possible biases.
In addition to those reasons, I also like pairing because it also reveals what kind of team player you are, and going to be if you're hired (are you just redirecting blame, are you taking any initiative, what kind of questions do you ask...). Which is one of the most underrated skills when hiring, IMO.
Then I'd ask them how they'd design the schema for that.
Then I'd ask them to sketch out the admin interface for managing users and how they'd implement that.
If they start asking questions about permissions and roles and such I'd be really impressed.
Essentially I'd act like a client/end user who kinda knows what they want to see if they can ask the right questions.
I'm not looking for a UX expert but more someone who thinks about the process and end result as well as a nice way to implement the code side.
I'd ask programming related questions as well but if (when) I hire I want someone who thinks about the end user experience while they are developing, I don't live in a world where a job comes with a 400 page spec and neither would they.
I take the view that I'd rather have someone who can think about how the end system will work than someone who knows ever single API for whatever language I'm hiring for, one of those you can find on google in 30 seconds the other not so much.