I don't use Chrome/Firefox but this surprises me, as well. The presence of a private browser window (anywhere) will prevent it from deleting cookies when you close this private browser window? That seems completely counterintuitive to me. When else, in any application, does the mere existence of an open, empty window completely change the behavior of a different window?
Safari's private tabs are each completely independent.
Safari's private tabs are each completely independent.