And another relevant question, is it actually happening more or are we just noticing it more and calling attention to it?
The concept of a hermit is very, very old. Some percentage of humans have always opted out of communal culture for a more isolated experience. Whether that percentage is rising, by how much, and why, is tough to analyze.
Also, the "why" of that question is ripe culture war fodder, which makes any discussion difficult.
>Some percentage of humans have always opted out of communal culture for a more isolated experience.
Society's perception of these people has varied over time too. At one point what we'd consider a shut-in might well have been considered a holy man in certain specific circumstances.
We reached the level of socio-technological development when a single person can happily live without a safety network, such as (extended) family or a community.
Why should one stay in the society, though? Was it ever intrinsically good? I mean slavery/serfdom, wars, racism, social inequality builtin in the culture multiplied by general narrow-mindedness. People love to ignore these inconveniences. The leader considered "the great" conquers the city and kills/rapes/enslaves everybody? Oh, that was normal. But now it's not, and it will never again repeat itself even on the smaller scale. Now society works perfectly. Yet fundamental physical constants and human biology remained the same. ¯\_(ツ)_/¯
>why are people dropping out of society in greater numbers than ever before? What changed?
Why not?
You grow being told that you go to school, you get the high grades, you get the good job, make friends, marry the girl, pop a few kids, and you're happy.
But it's never enough. There's always something you did wrong, always more you need to do right. You force yourself to do the things you're expected or you're forced to do the things you're expected to be happy about even when inside you're inside you scream and weep and want it all to just go away. Always more you must be doing it.
Hold on, they will tell you. Things will get better if you keep going, they will tell you.
You look at all the fighting at home. There's never any laughing or fun talk; you dread coming home because one wrong word or a wrong look and it's more shouting. This is what being with the girl you love is like? Turning home from a refuge into a place of fear and terror?
Hold on, they will tell you. Things will get better if you keep going, they will tell you.
Ultimately that turns out to be fine for you. You can't really get bring yourself to really befriend anyone anyways. You have nothing in common with anyone else. You have nothing you can say and nothing you can do that would make their lives just that little bit better. So you say nothing. And you do nothing. You just keep going on and on to the next thing and the next thing after that.
Hold on, they will tell you. Things will get better if you keep going, they will tell you.
But five years later on, and it didn't get better. Ten years on and it didn't get better. Twenty five years later and it never got better. On that day you realize what a sham it all was. It never got better, and it never will get better. They were lying to you all along. There is no better tomorrow.
And then you realize something else. Maybe that's fine to. The sun rises, other people laugh and play, and the world moves on whether you're there or not. So you just keep doing it. No hopes. No dreams. No future. Just never ending Now where you just survive the day, only to wake up and do it all over again until you don't.
So... after all that. Why not?
I realize that it might not help answer why more and more might choose to do so. I can't really offer answers to that. I can only answer why one person might choose to.
Honestly, the reason is really boring and it probably won't be fixed.
When an economy reaches stagnation/saturation people still want to play the capital accumulation game. So they start accumulating money and land that they never intend to use. Money is needed as a medium of exchange for the division of labor. Shelter is a basic need so people absolutely need land.
Because this is an accumulation game, owning more just means you demand more. If the interest rate (or gains from land or whatever financial asset) were 3% then some people end up unemployed because they cannot clear this artificial profitability gate. Employers want to hire whole individuals for as long as possible (ideally a 60 hour week). This means some people go without a job because 2 people working 60 hours can do the job of 3 people (on paper). People (especially the youth) are scared of ending up long term unemployed, so they skill up. They now have to compete with their peers for an artificially limited number of jobs. If they win the competition then they win big. If they lose, they lose big. The government must (they have no choice) intervene. Predictably, there must be an expansion of welfare. Remember how 2 people are doing the jobs of 3? Well, that means a small portion of that extra 1/3 will go to the person that ended up unemployed. It's extremely counterproductive but that is just how things are.
So now you have two options. Play the game, suffer, win and pay welfare for others or never play and get paid welfare (it's not like they want you as their competition anyway).
I have to put emphasis on the fact that the welfare is just a symptom to avoid a rise in extremism. If you get rid of it without treating the symptom then expect to see lots of angry people on the street who have nothing better to do than hate you.
The concept of a hermit is very, very old. Some percentage of humans have always opted out of communal culture for a more isolated experience. Whether that percentage is rising, by how much, and why, is tough to analyze.
Also, the "why" of that question is ripe culture war fodder, which makes any discussion difficult.