I was surprised to see that this tracking works across both regular and private browsing in Firefox (67.0.4 on macOS). I can see the number of visits increment and whatever message I've saved on either side is displayed to both.
It is creating a new session it's just that the server code uses a deterministic session key which is created based on your IP address and user agent. So as long as you use the same browser/IP combo you will get assigned the same etag, (at least user-agent is in there otherwise it would be extra trippy to load the page on FF and go to Safari and see the # visits/message displayed there too)
Private browsing mode wasn't (originally) intended to to prevent tracking by websites. It's goal was to keep your browsing history locally private. (i.e. keep your pr0n viewing out of the local browsing history so your friends wouldn't see what kind of porn you like when they borrow your laptop to check their email.
That definition has expanded a bit (very) recently, but preventing website tracking is usually a separate feature (adblockers, noscript, automatic cookie deletion tools, firefox's recent "tracking blocker").
Should only work for first party though. Safari partitions its cache by the origin in the address bar[1] so your typical advertising tracking should not work.