June 08, 2003
By Charles V. Peヨa
The Cato Institute
Total Information Awareness was a program built to run a massive
database to find would-be terrorists. Proving that government
programs never go away, the TIA is back, this time as the Terrorist
Information Awareness program. It's not even the same old program
dressed up in a new name. It's TIA as TIA. But if it was a bad
program before, it's a worse one now.
The Defense Advanced Research Project Agency, which launched TIA,
claims that the new TIA is designed "to protect U.S. citizens by
detecting and defeating foreign terrorist threats before an attack."
But the core of the new TIA is the same as the old: a database of
public and private records to be analyzed for patterns indicative of
terrorist activities.
TIA essentially depends on the "law of large numbers. It's what
marketing companies call "data mining." In a nutshell, "profiles"
are developed of people who should be good customers for a
particular product or service. A large pool of people who fit the
profile is targeted, knowing that only a small fraction will
actually be customers. That's what TIA will do. Only instead of
potential customers, the profiles will be for would-be terrorists.
And like commercial data mining, only a small fraction of the pool
of people who fit the profile of a terrorist will, in fact, be
actual terrorists.
So how hard or bad can that be? Well, a little "back of the
envelope" analysis helps shed some light on TIA. Assume a population
of 240 million adults (i.e., children are not would-be terrorist
candidates). Assume there are 5,000 terrorists lurking among us.
Assume a 99.9 percent probability (i.e., near perfect and very
highly unlikely) of correctly identifying a suspect as an actual
terrorist -- that is, if you suspect someone is a terrorist, he is
actually a terrorist. And assume a 99.9 percent probability (again,
highly unlikely) of correctly identifying a suspect as an innocent
person.
What's the probability of finding a terrorist hiding among the
masses? Without boring you with all the math involved, here's the
answer:
£About 244,299 people will be identified as suspected terrorists
(and remember that this is with near-perfect accuracy of being able
to correctly identify terrorists and innocent people).
£Approximately 239,995 innocent people will be mis-identified as
terrorists.
£The probability of finding a real terrorist is 2 percent.
The inescapable conclusion is that TIA, if broadly applied, is
simply a fishing expedition that casts a wide net, snaring many to
catch very few.
What happens if the accuracies for being able to correctly
predict whether someone is a terrorist or innocent person are
reduced from 99.9 percent to 90 percent (still very high and
unlikely)?
£Over 24 million people will be identified as suspected
terrorists.
£Slightly less than 24 million innocent people will be
mis-identified as terrorists.
£The probability of finding a real terrorist is .02 percent or
nearly zero.
That certainly doesn't bode well for the prospects of TIA.
But what if we reduce the total pool of potential terrorists to
just the U.S. Muslim population (not an entirely unreasonable
assumption given that al-Qaeda is a radical Islamic terrorist
organization) or about 6 million people and we split the difference
on accuracy to 95 percent (which is still very high and not a "real
world" statistical norm)?
£Almost 304,500 people will be identified as suspected
terrorists.
£Some 299,750 innocent people will be mis-identified as
terrorists.
£The probability of finding a real terrorist is 1.5 percent.
Still not very good, is it?
Here's one final example. Assume we're looking for 19 hijackers
within the U.S. male Muslim population, about 3.6 million people.
And assume near-perfect 99.9 percent accuracy. How hard would it
have been to find the hijackers and potentially avert 9/11?
£Nearly 3,619 people would have been identified as suspected
terrorists.
£And 3,600 innocent people would have been mis-identified as
terrorists.
The probability of finding a real hijacker is about one-half of 1
percent.
So when it comes to TIA, you do the math.
And here's one last cautionary note about TIA. Advocates are
claiming that it is a solution to "connecting the dots" of
intelligence data that weren't connected prior to 9/11. And, to be
fair, if the pool of people the TIA methodology is applied to is
small enough and focused (i.e., you already have good reason to
suspect someone might be a terrorist), it might be a useful tool.
But another major intelligence shortcoming was over-reliance on
technical intelligence gathering and not enough "old fashioned"
human intelligence. Revisiting TIA will be going "back to the
future" and falling into the same trap.
Charles V. Peヨa is director of defense policy studies at the Cato
Institute, www.cato.org.
|