This is a strange attitude; if you change jobs every three years, then how much are you going to get done, really? If everyone took this approach, then would any large project ever get finished?
Well, OP is talking about their first job, so my advice is targeting that specific situation. As a general rule, engineers see a fairly large bump in pay when going from their first to their second job, particularly if they did a poor job of negotiating their first salary. So yes, in terms of career development and realizing your earning potential, you should not stay at your first job for more than a few years.
In many startup product companies, the specific knowledge about the product is not interchangeable with other tech companies. The tech skills and the soft skills are, but a large part is know-how about the product and the market.
It makes sense to stick around longer if you see yourself as a "knowledge worker" rather than as an interchangeable techie. It certainly makes sense from the company's perspective to keep knowledgeable people around.