Had a family member that worked for SPEA (test equipment manufacture). Said different government organizations would bring in boards with massively parallel sets of chips and input/output on them, like nothing they ever saw in any other field. They were expressly forbidden to take pictures of them.
Between that and the massive data centers they are building I'm guessing they have rather impressive capabilities.
This particular set of exploits has little to do with collecting information. This seems to be directly related to command and control operations, including over systems that aren't connected to the internet.
There are a pretty scary set of discovered exploits.
I believe this is not the correct thread, but how can anyone sift through so much data, in general? Private companies need simpler things, like people you are likely to know in the real world, from the data they acquire. But intelligence agencies need actionable intelligence. That would require something way more intelligent than a simple spam filter.
That depends on the data you are talking about. The operations described here don't seems to collect huge amounts of data. If you're talking about the usual dragnet surveillance: a lot of it seems to be relatively simple filters and simple data correlation.
For example, you can build huge social graphs with simple metadata. Then you can search for all people who communicate a lot with people who communicate a lot with some known terrorist leader. Of those people, you take just those using tor. If any of them plans to enter the US, you flag them to be detained and searched at the airport. If any of them is already in the US, you can tell FBI to check them out.
Or you can look for sudden changes in message volumes. If terrorist leader A suddenly starts to communicate a lot more with random person B and random person C, who in term start communicating with other people, you suddenly have a whole list of people who might be planning a terrorist operation.
Of course you still need huge computing capacity even for these relatively simple operations, but they certainly have the funds for a few datacenters.
> intelligence agencies need actionable intelligence
For the most part that hasn't really been how it's worked so far. Generally intelligence agencies have used the information they've gathered so far to try to manipulate people.
'were', I think, may be more operative. I first heard that statistic about a long time ago, more like the '70s. Mathematicians don't seem to be very useful to the NSA's current hacker paradigm. (Note what we haven't gotten from the Snowden leaks so far: any sort of major mathematical or theoretical advance. Amazing hacking infrastructure, though.)
NLP's foundation is in statistics, so calling it "not maths" seems rather short-sighted. Mathematics, especially statistics, play a crucial role in all data interpretation when you get to any kind of scale and they seem to be the biggest of them all.....
Its only really useful against non-terrorists, but that doesn't mean its a waste of money: if it is used to make money.
So maybe its being used to make money - i.e. targetting non-terrorists (i.e. industrial espionage) - precisely because it is a huge waste of money. Wow, its almost like the whole thing was just a bad idea in the first place - its become self-serving.
Between that and the massive data centers they are building I'm guessing they have rather impressive capabilities.