Delicious
Digg
StumbleUpon
Newsvine
Facebook
Google
Yahoo
TechnoratiThe pilot programs ignored privacy safeguards, says a recent Homeland Security report.
From late 2004 until mid-2006, a little-known data-mining computer system developed by the US Department of Homeland Security to hunt terrorists, weapons of mass destruction, and biological weapons sifted through Americans' personal data with little regard for federal privacy laws.
Now the $42 million cutting-edge system, designed to process trillions of pieces of data, has been halted and could be canceled pending data-privacy reviews, according to a newly released report to Congress by the DHS's own internal watchdog. 
Data mining to help fight the war on terror has become an accepted, even mandated, method to provide timely security information. The DHS operates at least a dozen such programs; intelligence agencies and the Department of Defense employ many others.
But ADVISE (Analysis, Dissemination, Visualization, Insight and Semantic Enhancement) was special. An electronic omnivore conceived in 2003, it was designed to ingest information from scores of databases, blogs, e-mail traffic, intelligence reports, and other sources, government documents and researchers say.
Sifting that enormous mass at lightning speed, ADVISE was to display data patterns visually as "semantic graphs" - a sort of illuminated information constellation - in which an analyst's eye could spot links between people, places, events, travel, calls, and organizations worldwide.
Report: DHS didn't follow guidelines
Yet ADVISE, whose existence and scope were first detailed by the Monitor in February 2006, seems to have run afoul of its own ambitious scope. It failed to incorporate federal privacy laws into its system design. From its earliest days, the system's pilot programs used "live data, including personally identifiable information, from multiple sources in attempts to identify potential terrorist activity," but without taking steps required by federal law and DHS's own internal guidelines to keep that data from being misused, the DHS Office of Inspector General (OIG) said in a June report to Congress, which was made public Aug. 13.
In a rebuttal attached to the report, the DHS Directorate for Science and Technology disagreed with most of the OIG's findings. "The ADVISE tool set is little more than an empty framework to which data must be applied," wrote Jay Cohen, DHS undersecretary for science and technology, in a letter accompanying the rebuttal. He said no privacy laws were violated.
Even in searching for terrorists, data-mining programs are supposed to ensure that Americans' personal information is used only when necessary and lawful - and only for specific and proper uses. One problem is that even data that look anonymous aren't necessarily so. For instance, even when names and Social Security numbers are stripped from data files, programmers can still identify 87 percent of Americans through their date of birth, gender, and five-digit Zip Code, researchers say. So a system has to be carefully designed and use encryption and other computer techniques to comply with the law.
Last week the Pentagon shut down its TALON terrorism database program, which had been found to hold files on peace activists. In 2003, another military data-mining project - the Total Information Awareness project - was also ended following a congressional uproar over privacy fears.
Congress last fall ordered its Government Accountability Office to audit the program for privacy and effectiveness. It asked the OIG to do the same. In February, the GAO recommended a full-blown data-privacy review of the ADVISE system. Without that, its report said, ADVISE holds "potential for erroneous association of individuals with crime or terrorism and the misidentification of individuals with similar names."
In his report to Congress, publicly released earlier this month, DHS Inspector General Richard Skinner revealed that:
•From late 2004 to mid-2006, three ADVISE pilot programs - one focused on biological threats, another on weapons of mass destruction, and a third classified program to identify emerging threats - were not mere test beds working out technical bugs. Instead, they were "operational" and used "personally identifiable" data, without having conducted any privacy-risk assessments.
•All three pilot programs were quietly halted in March pending formal privacy impact assessments on the vulnerability of personal data. A privacy impact assessment is a type of information audit that ensures that government is only using personal information when it is necessary and lawful to be revealed.
•While submissions were made to begin the process, full-blown "privacy impact assessments" of the three programs did not begin until early 2007 - about two years after they became operational and began hunting terrorists, the OIG's office reported. It also said the March shutdown to assess privacy implications has damaged ADVISE's prospects, giving rise to skepticism within DHS about the utility and cost of the program and leaving it "at risk" of cancellation by 2008.
Failure to ensure data privacy is a problem that has torpedoed other counterterror programs, says Lee Tien, an attorney with the Electronic Frontier Foundation, a privacy advocacy group..
"The OIG's report clearly shows major breakdowns in the system we're depending on to protect people's private data," says Mr. Tien. "Whatever the data ADVISE used, the outputs are clearly important for people's privacy. The biodefense pilot program, for instance, presumably involves information about people's medical condition and emergency-room reporting."
Confusion within department's ranks
DHS's delay in addressing data privacy appears to be due to confusion and miscommunication about privacy requirements by ADVISE program managers and DHS's privacy office, amid the rush to get a system running, the OIG says.
For example, ADVISE program managers told OIG investigators they didn't realize privacy assessments were required for a system still in development. At that stage, the system was just a processing tool without data, they argued - a view agreed to by the DHS privacy office.
Indeed, the privacy office mentions the ADVISE system only once, in a footnote, in its mandatory report last summer to Congress on data-mining activities. Until the "ADVISE tool" had data attached to it, it was not a data-mining program needing privacy review, the office reported.
Unknown to the privacy office, the ADVISE pilot programs had been operational and using personal data for about 18 months before the privacy office made that report to Congress, the OIG found.
DHS has not reported how much and what type of personal information was used. One senior DHS official, who agreed to speak only on condition of anonymity, says of the personally identifiable data used by ADVISE: "We have no idea what information or how much was used."
Larry Orluskie, a spokesman for the DHS science and technology directorate, says a DHS privacy office review of ADVISE last month corroborates the OIG finding that ADVISE "was maybe too zealous in its testing," he says.
Even so, he says, the ADVISE system is back on track, though he is unsure if the privacy assessment was complete or if operations had resumed. A request for interviews with Undersecretary Cohen or other ADVISE officials went unanswered.
One conclusion, however, is that the privacy failure has cost ADVISE dearly. Despite some early successes - ADVISE's weapons of mass destruction pilot program identified a link between organized crime and terrorism - the failure to abide by privacy laws and costs of compliance have now reduced interest within DHS in ADVISE, the OIG reports.
Copyright © 2007 The Christian Science Monitor.
29 Comments so far
Show AllI think the claims that these programs are "effective" in some way should be challenged too. Similar to the idea that torture is effective, I suspect much of this activity is fueled by cartoon mythologies that just keep information technology contractors rolling in dough, offering little practical utility but tons of opportunity for abuse.
i am emitting biochemical toxins from my flapping butt cheeks right now...but not nearly as deadly as the nukular feces coming out everytime dumbya opens his mouth.
Is it true that Karl and Dick and others complained there was insufficient data to adequately test the search and download features of the software?
Maybe Chertoff can find the WH's 5,000,000 missing emails? He might look first in his coat pockets.
This is straight out of Orwell's 1984. THX 1138 deals with some of the same ideas.
There is nothing to fear, but fear of terrorism.
I heard it had to be shut down after in kidnapped and impregnated Julie Christie
"We are now all enemies of the state; we will learn to love the state."
Put all that data in the basement for future use.
Yabut, what'd they do with all the data? gonna throw that out?
So you shut off the computer, doesn't mean you still won't use the data.
Data, schmata. What do I care, I have my guns.
A year or so ago, in the Seattle Times, I believe it was, there was a brief article about a woman who wanted to make a short flight and found she was on a no-fly list. No one knew why, but she was grounded. Some attorneys later, it was found that, when talking to a friend on her cell phone, she said about her son, "Michael really bombed his test today."
Obviously, the word "bomb" = terrorist = no-fly. How many more are on various watch and no-fly lists from something as trivial and innocent is that? What would, "We are really planning to have a blast at the party tomorrow night," get you, I wonder?
With any database, GiGo (garbage in, garbage out)rules, but the results can be so inconvenient, if not tragic, these days.
I read an article today in the Times that reviewed all of the mistakes the CIA made before 9/11 and ended up by decrying the concern over the REALID program as a misguided populist plot and saying that we would be much, much, safer if we all had a government certified ID and would never again have to worry about a terrorist with a false ID. Sheesh!!!
There has always been a thriving industry in false ID's and I'm sure that anything the government comes up with, the mob, or somebody, will come up with a copy method which is near perfect. "What one man can devise, another can copy."
Anyone who has seen the latest Matt Damon movie, in the Jason Bourne series, got an eyeful (OK, maybe some poetic license ala Hollywood) as the CIA tried to track him across the globe. Early on, they pick up the code word -Blackbriar- and the spooks are hell-bent to trace who used that word, one of billions that are analyzed hourly across the global telecommuications "cloud". One wonders, if what we type here, or at Huffington, or any of the other progressive blogs, is analyzed by who knows who, and where, and its just a matter of time until enough "evidence" is pulled together for that hard knock at the door at 2am, and Blackwater's finest are standing there, and we are dragged out into the street in our PJ's and shoved into an awaiting black Suburban.
It was just a matter of time till they found so much dirt on the oligarchy that it decided not to take it any further.
Members of unions, environmental and peace organizations have already been treated or referred to as "terrorists." How can anyone prove that they have stopped? The way our intelligence is being farmed out to private corporations, does anybody imagine that they won't treat all of this as material they could use to make money? Merging corporate and government power, fascism, is not communism. But it it has the same result because communism merges it without corporations. Either way, the people lose their rights and a say so in their government. Greed of those in power leads to wars, environmental degradation and loss of freedom.
I'd like to know how many false leads are generated by such data mining, how much time and money are spent following up on them, and how many innocents end up on no-fly lists as a result.
Which of Donald Rumsfeld's categories does data mining fall into: Known-unknowns or unknown-unknowns? Or doesn't anybody even know?
And for the fun part. The information and so-called 'leads' from this get widely shared. That was another of the cute little changes made in the Patriot Act. Before, 'intelligence' information was rightly known to be different. It might be rumor. It might be gossip. It always took an analyst to look at it and see what it might mean.
The Patriot Act broke down the wall and now any 'intelligence' info can be shared with all sorts of police agencies. Especially through the "Joint Terrorism Task Forces" that exist almost everywhere in the US.
So, when this cute little program comes up with a false positive pointing that you might be involved in biological weapons, odds are that this gets shared right down to the local cops. Of course, they won't know its a dubious lead from some experimental program. You just find yourself getting the shit kicked out of you by some cops who keep calling you a 'terrorist' because there's now an entry in your police file that says that DHS thinks you are involved with biological weapons.
And even better, some of these lists and this information is being used for employment decisions, or by credit bureaus. Get your name on a 'watch list' because some experimental program listed you as a false positive and suddenly you can't get a job.
What I hate is never hearing about the survailance program before then
Isn't this the system that automatically picks out your conversation when you say the right word on a phone, or in an email? Just think, sites like CD must be creating a lot of jobs these days :-)
I wonder what the name will be when resurrected?
Remember TALON which was also found to be illigal?
"In February, the GAO recommended a full-blown data-privacy review of the ADVISE system. Without that, its report said, ADVISE holds "potential for erroneous association of individuals with crime or terrorism and the misidentification of individuals with similar names.""
Sounds like the same inadequacies discovered with "No Fly List".
And who is watching the watchers who are watching the watchers when nobody is watching...
Or just sick and tired of watching and need a breaking from all the watching????
Face it, the problem here is not using this type of system to locate terrorist cells but the abuse that this system facilitates. Given the track record of the current "less than scrupulous" administration the information obtained by this system could easily be used for political purposes. And who would know? All the president has to do is ignore subpoenas and declare "executive privilege".
And what of a future even less scrupulous administration? Once this becomes common practice it is merely accepted without question.
All things considered, this is one more brick in the wall that becomes the world described in the George Orwell book "1984" as Big Brother keeps tabs on everyone, not just those who would do harm.
as someone who worked for an intelligence dept. in the US there are approx. 2oo undercover police, special ops. of US intelligence dept, DEA,FBI and approx. 10 other intelligence depts. that go deep cover living in our neighborhoods collecting data, most written for their agendas, for example when congress hands out money for the budget, just to collect data and then act upon it whether it's true or not, similar to KGB in Russia. Right now we are living in a fascist society, heading towards communism. wake up NOW PEOPLE or it will be too late. the military will be on every corner of every street building cement walls around every neighborhood and demanding identification and where you are going and why, that is, if you don't live there. we are heading towards the situation in Iraq, which is a training mission for the invasion of US's neighborhoods, all in the name of security.
I just love learning about some data mining program is being shut down or it's activities questioned and even suspended for review.
What I hate is never hearing about the survailance program before then.
This just makes me wonder what else is going on which evidently nobody knows about except the people who are doing it.
Add privatized intelligence and psy-ops for hire and ...?
But how come nobody mentions the abuse of this personal info as it may apply to insider trading? Or other illegal activities where such detail into personal lives can be useful to the corrupt or criminal? Why is it that insider trading never becomes an issue when phone conversations can be monitored?
Who watches the watchers when nobody is watching?
semantic graphs...linking words like...bomb, which also means a theatrical or other performance failure...and doesn't account for metaphorical language or hyperbole, common everyday language usages, such as, "I'd like to strangle him" or "fucking blow 'em all to hell for all I care." Personally, I think shooting George Bush would be a waste of a good bullet.
Ragdoll - So why wouldn't the Bin Laden gang and its multiple branches use carrier pigeons now?
Bush thought of that too (Bin Laden using carrier pigeons). They had to close that operation down also - being that Cheney was such a bad shot.
Setting aside the potential for blackmail, insider trading, election rigging and other potential misuses of these systems by an administration whose dirty tricks have put the country to shame, can we recall that boxcutter "WMD's" that were the only weapons used on 9/11?
So why wouldn't the Bin Laden gang and its multiple branches use carrier pigeons now?
And you know - no matter what they say, they will
not destroy the data they have already collected.
It is backed up multiple times and in multiple
locations they probably don't even know about
(until they want to look at it again.)
Wandering in a dark forest of heavily encrypted acronyms,
We the People seek the narrow winding path
That leads back to light and freedom.