I take a guess that you are using an HDA based sound chip? If yes, have a look at [1]. Can't help you out more than that though, it's been years since I was forced to deal with that crap on Linux.
If it's any consolation... I recently came across an ASUS all-in-one PC where it was not enough to install the Realtek Windows HDA driver, no... that would have been too easy. You also needed to install a tiny extra driver that wasn't visible by default on their driver list in the Audio driver section. The entire audio world is utter, utter madness.
Huh. No issues like that at all with any of my Linux desktops and analog jacks (mostly Devuan). They all seem to detect and maintain sources fine.
The problem I personally have (if there are any pulseaudio experts out there) is it continually re-enabling ephemeral audio sources like HDMI and bluetooth when they are plugged back in, even if I'd set them to "off" before unplugging them - fortunately since I set the analog to default it usually doesn't mess anything up, but it's a bit untidy, and also annoying for the one USB webcam that has a mic plugged in, where I get a surprise input if I don't watch it.
It'd be great to have it persist settings for identically named sources even after they are removed.
Try removing pulseaudio, if only temporarily. I know pulseaudio is one of the few linux things that actually does things based on headset jack detection. For all I know the driver on your system might be broken, or sending spurious inputs.
Also had that issue. The 3.5 would work if it was plugged in just the right amount. If it was bumped it would disconnect. In a similar issue, my Nintendo switch controllers often disconnect while in the plugged in mode because the connectors just lose contact sometimes. I wish I could tell it to use them in bluetooth mode and charge only over the connector because bluetooth is pretty much flawless.
Sounds like you pushed the wrong button. If you can't even keep your 3.5mm audio running there's little chance you can work Bluetooth with all its foibles.
If it's any consolation... I recently came across an ASUS all-in-one PC where it was not enough to install the Realtek Windows HDA driver, no... that would have been too easy. You also needed to install a tiny extra driver that wasn't visible by default on their driver list in the Audio driver section. The entire audio world is utter, utter madness.
[1] https://www.kernel.org/doc/html/latest/sound/hd-audio/notes....