Jitsi Meet is not perfect, but better about this. It doesn't make a P2P mesh (where I have the same experience, parts will always fail), but instead WebRTC connections to central servers that distribute the streams, so each participant has only one upstream. They have a public testing instance on https://meet.jit.si/, but running your own is better.