That’s quite a big assumption to think that it would be clearly labeled, and also that it would have a dedicated fuse. It’s not like that would be such a huge power draw that it needs its own fuse. Pulling the fuse would likely cause the whole infotainment system to go down.
Reading back at my reply, I didn't mean it to sound so authoritative. Just an idea.
I helped a friend do this a couple years ago in a fairly modern car. It most certainly was not clearly labeled, but there were clues that led us to try it out. It indeed disabled more than just the cellular modem but the goal was to disable all "smart" features altogether, along with anything that collects telemetry or phones home. After monitoring how it affected the car, the mild trade-offs were deemed worth it.