I nuked my LinkedIn a couple of months ago. I had the same fear, but I decided fear is is not a good guide. Looking at it rationally, I never got a position through LinkedIn and I am very happy with the positions that I got so far.
If I ever end up in a world where it's only possible to get a job using LinkedIn, I can probably recreate my profile, along with contacts, and recommendations in a week or so.
What's funny is the same thing works in reverse but not enough to deter completely : "what (real life) shenanigans am I attracting by hoping this isn't a total timesuck?"
Clients from LinkedIn in are way more high maintenance on average compared to StackOverflow/SE and GitHub. Maybe it's not as bad for recruiters.
Easier is disabling your profile. You keep your contact network and but no longer appear in search results and can only be viewed by people in your network.
If I ever end up in a world where it's only possible to get a job using LinkedIn, I can probably recreate my profile, along with contacts, and recommendations in a week or so.