What bugs me is the fact that much of the public thinks that the NSA conducts mass surveillance. It's not practical or particularly fruitful to keep track over 300 million US citizens when in actuality, they trace threads from certain countries (mainly in the middle east) to back home. That's like expecting police to patrol every street corner in riot gear and carrying assault rifles.
Bullshit.
As a software developer, I will let you know that it is
extremely easy. Like, not even "we need a team for this" easy, but as in "I could personally write software capable of doing most of this reasonably well if I set my mind to it, despite that part of software not being my focus." Increasing the numbers
actually makes it even easier, assuming you have a server farm to do the computations on. The boundaries to this work are not grounded in technology, but rather in policy. Quite frankly, I would consider them highly incompetent if they
didn't have such capabilities, given the policy directs them towards doing so.
If you want an example of a similar system, look at IBM's Watson. In terms of code, it's actually incredibly similar. Learning algorithms + large databases + associative learning is incredibly powerful and can be applied to all sorts of data. For example, Watson's latest application is healthcare diagnosis; and is currently more accurate than doctors.
And the big problem is, everyone has something to hide. You, me, everyone. You've either broken some obscure law or at the very least have been associated with something which could either ruin your life or convince a jury to convict you for something. With access to those databases in an easier searchable format; as you would get with such AI techniques as they are using; you could find such tidbits as 'this person looked up nerve gas, dirty bombs, videos of explosions, nuclear weapons, and certain chemical weapons. They also talked to persons x, y, and z, who [at some point in the past] looked up [list of similar topics].' Never mind the fact that most people occasionally look up such things when wiki-walking or reading about history; if you presented cherrypicked evidence like that, you could convince a jury that just about anyone was a raving loony out to kill everyone.[/list]