From Waipapa Point[1] on Te Waipounamu, if you were looking west or south-west towards Rakiura, you'd be looking at the "Tasman Sea". Otherwise the waters are the "South Pacific Ocean". See sections 61 and 63 of [1] for more detail.
Yeah as a fellow Kiwi (living in Australia) I've always considered the Southern Ocean to start somewhere around the subantarctic islands (Auckland Island, Campbell Island, and Macquarie Island) below NZ.
[1] https://www.openstreetmap.org/node/1074610331
[2] https://iho.int/uploads/user/pubs/standards/s-23/S-23_Ed3_19...