I'm surprised there's a shortage, I picture being a pilot as a very desirable career that would lead to an oversupply. Is it because learning to fly is too expensive for many of the people who want to do it?
That's part of it. The US airline industry expects pilot candidates to apply when they already have the necessary FAA licenses and flight hours. To reach that point, pilots have to either serve in the military for 7+ years or take on a huge amount of student debt. If the airlines had instead maintained a pipeline by hiring trainees and then paying them to get certified then they wouldn't have such a severe shortage now.
Airlines have also laid off pilots during several industry downturns since 2001. Every time that happens, some switch careers and never return to flying.
Then add in the large amount of away-from-home time, non-trivial danger, and post 9/11 rules, and it's easy to see why a lot of folks are profoundly 'meh' about it.
Know a few former Naval Aviators who hit their 8-10 year commit and decided to get a Masters in something not aviation related -- work-life balance was a definite draw.
Airlines have also laid off pilots during several industry downturns since 2001. Every time that happens, some switch careers and never return to flying.