[sinneduy]

At my current workplace, this also wouldn't fly. Its just not worth it in the long run, especially when your company is small (from a managerial perspective)

That being said, obviously your manager doesn't share this ethos. In the future, this is something you might want to filter for.

If I were you, I would make preparations for the worst and escalate. You don't want to leave, but you need to be willing to and prepared to do so. If you do escalate though, go to your manager. He's clearly aware of it, and ultimately its politically a mistake to go to either HR or your manager's manager, since it'll hurt your relations with your direct manager.

Doing nothing isn't an option in the long term if nothing changes. There's no real point in considering it.