[minouye]

This is a hard problem. And it makes a great interview question too!

Recent PM interview question I've been using: You're the PM at Netflix handling the home screen. How do you determine how shows get promoted editorially vs algorithmically recommended? Walk through metrics/principles/trade-offs and how it impacts various parts of the biz.

https://twitter.com/sriramk/status/1222547047846297600

[extr]

This is a great question and I might steal this for future interviews. It's surprising how many of the people responding, despite being told optimizing for engagement is a cop-out, still jump to some kind of metric-based prescription. Or just jump right into throwing percentages for each content type (I mean, it's twitter, so sure, have a go).

My personal stab at this: this is really a strategic question. What are Netflix's long term goals? To be seen as HBO is, as a content-creator, or merely a content-provider? Both? What are the typical engagement rates for Netflix content vs purchased? What is the real, intangible (unmeasurable) objective we are maximizing for? Total hours watched? Or perceived value by consumer? Someone spending 25 hours a week with Netflix on still might perceive it as lower value than HBO that they spend 2 hours a week watching (and thus be more likely to cancel). Some forms of entertainment are easily substituted, as someone who uses TV for background noise wouldn't particularly mind to use Hulu for that purpose instead of Netflix. But a "must watch" show on HBO is just that, a must watch. Which customer do we want more of? More "background noise" customers implies a marginally higher infrastructure cost, while a more content focused strategy implies a higher cost for content production (and possibly a much more variable revenue stream, as people subscribe/cancel as their favorite show starts/stops airing). Maybe we want both, and we want to identify what kind of watcher a customer is, and then tailor their home screen to suit. Maybe one person has 90% Netflix Originals and the other has 10%.

If there is an answer here, it's to carefully weigh these strategic objectives and only then make changes to show promotion. Even then, it would be important to have some system of monitoring in place that allows you to confirm your changes are actually making measurable impact the objective in question (this may be very difficult, given an intangible goal of something like "increase Netflix mind-share". Probably a lot of marketing surveys and focus groups.) Straight-up engagement rate is just one of the many things to track here, and increasing it at any cost might not even be in Netflix's long term best interest.

[minouye]

You're hired! Great strategic answer that gets to the heart of the question--it's less about the solution and more about how you frame the problem and the underlying goals and assumptions.

Do you work in this space? It seems like you have a lot of context around OTT/streaming.

Your answer also brought to mind this podcast episode I came across this morning: http://investorfieldguide.com/shishir-mehrotra-the-art-and-s...

It introduces a nice concept of "marginal churn contribution" framed with bundling, but I think is also relevant to this discussion. Maybe more on the content sourcing/production side, but bleeds over into long-term goals (reducing churn/maximizing ltv/etc).

[extr]

Thanks! I actually don't work in streaming/OTT, I manage a data science team for an auto insurance company. We end up thinking a lot about these kinds of questions from different business units that aren't quite sure what they want to be optimizing for, so I'm somewhat used to making the connection between high level strategy and measurable objectives. And as you might imagine churn/bundling/LTV are also very important concepts in the insurance space, which probably gave me a leg up in my response. The revenue model is at least superficially similar.

Thanks for the podcast link, it looks very relevant and will give it a listen. I see insurance is even explicitly addressed @ the 23 minute mark.

[maps7]

I'm not a PM nor hiring one but in my opinion you didn't answer the question. If the aim of the question is just to not answer the question and just raise more questions then it's a poor interview question in my opinion.

And from looking on Twitter, it's confirmed, imo, it's a bad question:

"As a lot of you have figured out instantly - this is really a iceberg question. Looks like a simple UX problem but is really about trying to understand Netflix as a system and as a business."

[adjkant]

> is really about trying to understand Netflix as a system and as a business

Why doesn't that make it a great question? A PM should be trying to do exactly this, yes?

[maps7]

No. Ask a question directly about it then.