I've worked at some of these places in my career. If you want to be surrounded by smart people all the time, pick the labs. Note: if you don't have a PhD you will be made to feel inferior to everybody else you work with, it just comes with the territory. The Non/Not-Profits can be friendlier places to work. Go for an analyst job or something similar. Most of those are also full of really smart folks, and making $$$ isn't the most important thing in the world for them. Many labs are managed by these non-profits. I listed the Contractors because it's easier to get analyst type positions with them, but I've found the general caliber of the people there (in terms of pure "smartness") to be lower. It can be tough to go from a Lab like environment, where your next cube co-worker speaks three languages fluently, can fly six kinds of aircraft, holds two PhDs and built a small-scale particle accelerator in his garage to relax to a contractor where your next cube co-worker debates, heatedly, between diet-coke and diet-pepsi, and how to score that cute waitress in the happy hour bar without his wife noticing.
Could you give some examples of the kinds of analyst/research work people without PhDs do at these places? What kind of "formal background" do you need to get them to even consider you for one of the more thinky analyst/researcher positions?
I've looked before at some of the places you mentioned, but it seemed extremely difficult to get someone to even talk to you about the interesting analyst/research positions if you didn't have a PhD.
My background: For the longest time, I thought I would go to grad school and become a professor in pure math. I love doing research and don't care about money...but I hated the whole bureaucracy and lack of real freedom (until you're a tenured professor) of it all. I really miss having a research environment, though, so like the OP I've been wondering for a while how to get back into this.
Sure, I know that this project http://starlight.pnl.gov/ was a decade long R&D project with a staff of zero PhDs. The NVAC ( http://nvac.pnl.gov/ ) place is peopled by lots of non-PhD hacker types doing cool things. Many of the better hacker types I've run across are also not from traditional CS backgrounds which is interesting, lots of trained physicists and mathematicians.
The non-profit types have loads of people who are not-PhDs who do clever think-tank style work for the government. Lots of report writing mostly, but if you can get latched onto some IRAD type project and do internal R&D work, you can see some really cool stuff -- I'm pretty sure I've seen the first ever real-time fusion of geospatial data from six different imagery platforms in a 3d environment. But even the report writing can be interesting, things like open-source studies of countries, industries, facilities, etc. But they can also be a back door into the lab work if you work for one of the managing organizations (like Battelle). Look into them closely as they also tend to have fairly large internal R&D departments. It's possible to make a career working on IRAD projects and responding to SBIR grants. Both are pure research.
Other places do things like geospatial systems research, text mining, information visualization, network security research (which for lab purposes usually requires lots of high level systems understanding or solid math, not as much low level hacking as you might expect, lots of the work is in predictive theories and social network analysis), pattern detection, image analysis, image change detection -- lots of good places for applied math, acoustic triangulation, emergency response modeling and simulation, etc. Think DARPA style research, which the labs do lots of work for.
If you are into hardware, there's also plenty of that, communication systems, weapons testing (like naval guns), hardware encryption systems, phased array radars, biochem detection systems, rapid prototyping, radiation detectors etc.
Most of these don't require PhD backgrounds, and if you hit all the ones you are interested in, you might find something that sticks.
Honestly, given your background, you'd probably find a better home at the labs than the companies. But the hiring cycle is abysmally slow, don't take a non-response as evidence of a lack of desire to hire. 6-12 month wait times on call backs are not uncommon. Often followed by 5-20 years of happy employment.
I personally found both the lab and non-profit environments a bit too rarefied for my tastes. But I do miss being surrounded by brilliant people every day. Also, keep in mind that some of these are enormous organizations, Google-sized by way of comparison, their campuses can be measured in square-miles, not acres and they often have tens of thousands of employees. If you don't live in the States (or aren't a citizen), there are probably equivalent organizations in your home country. Often there is lots of cross-pollination as well, I worked on projects with the Brits, the Italians, Singapore and the Aussies.
This is a good list: http://en.wikipedia.org/wiki/FFRDC also look at the managing organizations.
Non/Not Profit Companies
Battelle Memorial Institute - https://www.battelle.org/ Noblis - http://www.noblis.org/ Mitre - http://www.mitre.org/ Aerospace Corporation - http://www.aero.org/ Rand Corporation - http://www.rand.org/ CNA - http://www.cna.org/
Some for-profit Government Contractors (I've tried to pick ones that don't make stuff like planes and tanks, they focus more on thinkery)
SAIC - http://www.saic.com Bechtel - http://www.bechtel.com Mantech - http://www.mantech.com CACI - http://www.caci.com TASC - http://www.tasc.com
there's actually a whole ton of smaller ones like Blackbird Technologies ( http://www.blackbirdtech.com/ ) that you can make a good home in also.
Labs (don't let their Manhattan Project backgrounds fool you, they do tons and tons of R&D outside of nukes, great places with lots of smart people)
Lawrence Livermore - http://www.llnl.gov/ Pacific Northwest - http://www.pnl.gov/ Los Alamos - http://www.lanl.gov/ Brookhaven National Laboratory - http://www.bnl.gov/ National Renewable Energy Laboratory - http://www.nrel.gov/ Sandia National Laboratories - http://www.sandia.gov etc.
I've worked at some of these places in my career. If you want to be surrounded by smart people all the time, pick the labs. Note: if you don't have a PhD you will be made to feel inferior to everybody else you work with, it just comes with the territory. The Non/Not-Profits can be friendlier places to work. Go for an analyst job or something similar. Most of those are also full of really smart folks, and making $$$ isn't the most important thing in the world for them. Many labs are managed by these non-profits. I listed the Contractors because it's easier to get analyst type positions with them, but I've found the general caliber of the people there (in terms of pure "smartness") to be lower. It can be tough to go from a Lab like environment, where your next cube co-worker speaks three languages fluently, can fly six kinds of aircraft, holds two PhDs and built a small-scale particle accelerator in his garage to relax to a contractor where your next cube co-worker debates, heatedly, between diet-coke and diet-pepsi, and how to score that cute waitress in the happy hour bar without his wife noticing.