low level programming knowledge is fundamental if you want to work with security research (software vulnerabilities, reverse engineering and exploits) and the job market at the moment is quite big.
If not for employment opportunities, you should be (gently) exploiting these connections for educational value. They're in the industry and have likely seen examples of colleagues transitioning in from web/backend development. At the very least, they can easily tell you what mistakes to avoid.
I often have the same mindset you do about exploiting connections, and although it's been a struggle for me to change my habits, using people as educational outlets has been something that I've found to be extremely helpful to me personally and also often enhances my relationship with that individual as well.
It's something I dabble in and think I'm getting quite good, but unless I start publishing/presenting, I don't obviously see it leading to paid work.
I have a lot of long-time friends in InfoSec but don't really want to exploit those connections to look for work (yet).