CREW Posts Details and Copies of the Missing White House Emails Released by Administration

Nearly two years after Citizens for Responsibility and Ethics in
Washington (CREW) sued the Bush White House for both its refusal to
restore the millions of missing White House emails and its failure to
put in place an effective electronic record keeping system, the White
House has finally released documents that support CREW's allegations.
The documents, released after negotiations with the current
administration, represent only a small percentage of the promised
records, and appear to be part of a set of documents already provided
to the House Oversight and Government Reform Committee in 2007 and 2008.

These documents confirm Bush White House officials knew they were
failing to properly archive records and made several attempts to
develop an email archiving system. Although some officials described
the development of such a system as a "number 1 priority," the efforts
were either unsuccessful or abandoned for unexplained reasons. The
documents make clear some administration officials were aware of the
problem as early as February 2004, when the White House was attempting
to respond to an unidentified grand jury subpoena from the Justice
Department.

The documents confirm that in October 2005, the White House
discovered millions of emails had disappeared. The documents also show
that emails Special Counsel Patrick Fitzgerald had subpoenaed in
connection with the Valerie Plame Wilson leak investigation were
missing from Vice President Cheney's office.

In providing these documents to CREW and the National Security
Archive (which brought a separate lawsuit now consolidated with
CREW's), the Obama administration marked some of the documents
"sensitive," and therefore not subject to public disclosure, and
redacted the identities and contact information of virtually all
individuals named in the documents.

Many questions remain and the White House has promised to release
more documents shortly. For example, there are approximately 38 boxes
of documents the administration plans to review for disclosure. These
boxes contain records related to the White House's discovery of the
missing email problem as well as proposals to address the issue and
implement effective electronic recordkeeping. CREW is also awaiting
documents regarding the limited effort to restore some of the missing
emails that was begun by the Bush White House and is continuing. CREW
anticipates these additional documents will fill in more of the blanks
and will inform the public whether the White House is finally on the
right track with its electronic record keeping practices.

The documents released so far address the following key subjects:

* The Bush Administration's repeated attempts to develop a system to
archive Microsoft Outlook emails. In 2002, the White House began
converting from Lotus Notes to Microsoft Outlook/Exchange for its
email, and needed a system to preserve the Microsoft-based email. The
documents include proposals and a "statement of work" for a pilot
project to make the emails compatible with the Automatic Records
Management System (ARMS), the system used since 1994 to preserve White
House emails. OAP00000011, OAP00000025, OAP00000040, OAP00000056. But, as other records confirm, this system was never fully built. OAP00000083, OAP00000399.

* The Electronic Communication Records Management System (ECRMS), a
plan to develop a longer-term solution for email. The contracts include
one for Booz Allen Hamilton, which was selected to build the system. OAP00000386.
For unexplained reasons, the Department of the Interior's Mineral
Management Service issued the contract documents and received all
invoices, even though the system was to be delivered to the White
House's Office of Administration (OA). OAP00000386.
One May 2006 email discusses plans to put ECRMS into production,
stating ECRMS was OA's "number 1 priority," and "the most important
system that we have implemented in a long time, we need to get it
right." OAP00000719. Yet the Bush White House never implemented this plan as well.

* The Bush Administration was aware of problems with email
preservation at least as early as February 2004. The documents include
a "post-mortem" analysis by Microsoft of problems with searching for
emails in response to a January 2004 grand jury subpoena that stated
"there is no current mechanism to transfer Exchange email into ARMS,"
and the plan for doing so "is not yet a stable and consistent solution"
that "fails to consistently" move data into ARMS. OAP00000083
.

* The White House's discovery that millions of emails were missing
from its electronic files, and its attempt to address the problem,
comprise the bulk of the documents. They confirm the White House's
discovery of an investigation into the missing emails in October 2005.
Several documents from that period reflect a tabulation of White House
email files, called PST files. Those documents indicate there were a
total of 5,397 PST files, but 20 files were "missing" and 19 more were
"empty." OAP00000500.
The White House was unable to determine which White House component
(i.e., the Office of the Vice President, the Office and Management and
Budget, and the Council on Environmental Quality) was associated with
more than 1,000 files. OAP00000486, OAP00000500.
Others emails indicate PST files from August 10 through October 4, 2005
for separate White House components had been combined into a single
file due to an "inadvertent" change. OAP00000167.
Perhaps as a result of these problems White House appears to have
temporarily stopped creating PST files for at least a week. OAP00000379.

* The Office of the Vice President (OVP) had particular problems
with missing emails. Several of the documents discuss copying and
conducting a manual review of more than 200 PST files from OVP. OAP00000377, OAP00000741, OAP00000790, OAP00001411, OAP00001415.
This review may have led to the creation of a spreadsheet compiling
information about the OVP PST files, which showed gaps in the dates of
preserved messages preserved. OAP00000778.
One of those gaps was from at least October 1-3, 2003, a period for
which Special Counsel Patrick Fitzgerald sought email during his
investigation of the leaks that led the disclosure of Valerie Plame's
identity. OAP00000377.

* The documents show that during this period, the White House
initiated a "component by component analysis" to find the missing
emails. OAP00001387.
This apparently resulted in a series of scans that output the total
number of messages in each PST file, and the number of emails from each
date. OAP00000803, OAP00000842, OAP00001096, OAP00000903, OAP00000999, OAP00001158, OAP00001173, OAP00001176, OAP00001179, OAP00001182, OAP00001192, OAP00001229, OAP00001314, OAP00001350.
One email states every source file was covered, except for 30 OVP files
(it is not clear why), with 19 files that had no message count. OAP00001407.
In February 2006, the White House completed an analysis of all the
missing emails, and concluded there were 473 days on which there were
no messages preserved, and 229 days on which the number preserved was
suspiciously low. A chart of this analysis was released by House
Oversight and Government Reform Committee in February 2008. https://oversight.house.gov/documents/20080227155329.pdf. It is not clear if the documents provided by the White House to CREW and NSA were used in the creation of this chart.

* Some of the problems with the email preservation system were
summarized in a November 14, 2005 memorandum from Steven McDevitt,
Director of the Architecture and Engineering Directorate, to John
Straub, Acting Chief Information Officer. OAP00000399.
(An unredacted copy of this memo was published by the House Oversight
and Government Affairs Committee, but the one provided by the White
House blacks out the names of Mr. McDevitt and Mr. Straub in most
places.) The memo states the "current email archive process depends on
manual operations and monitoring, standard operating procedures do not
exist, automated tools that support the email archive process are not
robust, and there is no dedicated archive storage location. As a result
the current process and lack of storage management limitations result
in potential loss of emails. Lost or misplaced email archives in turn
result in an inability to meet statutory requirements."

* Mr. McDevitt was attempting to obtain approval for a new standard
operating procedure for archiving Microsoft Outlook/Exchange email, and
implementation of ECRMS as a long-term solution.

* The increased scrutiny may have led to the discovery of other
problems. For instance, one email discusses "unauthorized actions" that
were still taking place, OAP00000374,
and another asks for an "emergency change" to allow a program to be run
daily that collected attributes of files so that PSTs can be monitored
and tracked better, OAP00001413.

* A few documents from January and February 2006 appear to relate to
the recovery of some OVP PST files that needed to be searched. One
email instructs the recipient to being the three phase project for
restoring OVP email from 14 or 15 days in December 2003 and
January-February 2004. OAP00001392.

* In June 2007, the White House issued a "Request for Quote" to
install, configure, and test a new pilot electronic records management
system using a product made by EMC. EOP0000127.
The proposed contract calls for an "aggressive" six-week time frame for
the project, which would not include rolling out the system for all
1,800 users. CREW understands this system was not implemented during
the last administration. In addition, an undated "limited source
justification" appears to contemplate awarding a sole source contract
to EMC to acquire software and to configure, implement, and install an
"enterprise wide Records Management (RM) system." EOP0000156.

* Other documents provide a limited amount of information about the
present system. One, titled "EMC Messaging - Messaging Current Product
Compatibility Guide," describes certain email preservation products and
their compatibility with other software. EOP0000227.
Another is a "Request for Information" to obtain information about the
deployment of "an operational and improved electronic Records
Management (eRM) for email-records program for The Executive Office of
the President." EOP0000285.