September 23, 2005 | Advisory Panel Report Made Public

The Secure Flight Working Group, a set of security and privacy experts that was tasked in January with evaluating Secure Flight for the Transportation Security Administration's Aviation Security Advisory Committee, has released its report, which questions whether recommends that Congress stop any live testing until the TSA has adequately developed a plan for a new system to screen the nation's air travelers.

The TSA published the report -- perhaps inadvertently -- to their website this morning.

Despite having security clearances and signing NDAs, the group was left with more questions than answers and recommended that major changes happen before any operational testing, such as that proposed by the TSA to start in October.

The group consisted of

• Jim Dempsey, Center for Democracy & Technology/Markle Foundation


• Bruce Schneier, Counterpane Internet Security


• Steve Lilienthal, Free Congress Foundation


• Martin Abrams, Hunton & Williams


• Daniel Gallington, Potomac Institute


• Edward Felten, Princeton University


• Linda Ackerman, Privacy Activism


• Anna Slomovic, SRA International


• Lauren Gelman, Stanford University

They wrote:

II. Questions

The SFWG found that TSA has failed to answer certain key questions about Secure Flight: First and foremost, TSA has not articulated what the specific goals of Secure Flight are. Based on the limited test results presented to us, we cannot assess whether even the general goal of evaluating passengers for the risk they represent to aviation security is a realistic or feasible one or how TSA proposes to achieve it. We do not know how much or what kind of personal information the system will collect or how data from various sources will flow through the system.

Until TSA answers these questions, it is impossible to evaluate the potential privacy or security impact of the program, including:
􀂃 Minimizing false positives and dealing with them when they occur.
􀂃 Misuse of information in the system.
􀂃 Inappropriate or illegal access by persons with and without permissions.
􀂃 Preventing use of the system and information processed through it for purposes other than airline passenger screening.

The following broadly defined questions represent the critical issues we believe TSA must address before we or any other advisory body can effectively evaluate the privacy and security impact of Secure Flight on the public.

1. What is the goal or goals of Secure Flight? The TSA is under a Congressional mandate to match domestic airline passenger lists against the consolidated terrorist watch list. TSA has failed to specify with consistency whether watch list matching is the only goal of Secure Flight at this stage. The Secure Flight Capabilities and Testing Overview, dated February 9, 2005 (a non-public document given to the SFWG), states in the Appendix that the program is not looking for unknown terrorists and has no intention of doing so. On June 29, 2005, Justin Oberman (Assistant Administrator, Secure Flight/Registered Traveler) testified to a Congressional committee that “Another goal proposed for Secure Flight is its use to establish “Mechanisms for … violent criminal data vetting.”2 Finally, TSA has never been forthcoming about whether it has an additional, implicit goal - the tracking of terrorism suspects (whose presence on the terrorist watch list does not necessarily signify intention to commit violence on a flight).
While the problem of failing to establish clear goals for Secure Flight at a given point in time may arise from not recognizing the difference between program definition and program evolution, it is clearly an issue the TSA must address if Secure Flight is to proceed.

2. What is the architecture of the Secure Flight system? The Working Group received limited information about the technical architecture of Secure Flight and none about how software and hardware choices were made. We know very little about how data will be collected, transferred, analyzed, stored or deleted. Although we are charged with evaluating the privacy and security of the system, we saw no statements of privacy policies and procedures other than Privacy Act notices published in the Federal Register for Secure Flight testing. No data management plan either for the test phase or the program as implemented was provided or discussed.

3. Will Secure Flight be linked to other TSA applications? Linkage with other screening programs (such as Registered Traveler, Transportation Worker Identification and Credentialing (TWIC), and Customs and Border Patrol systems like U.S.-VISIT) that may operate on the same platform as Secure Flight is another aspect of the architecture and security question. Unanswered questions remain about how Secure Flight will interact with other vetting programs operating on the same platform; how it will ensure that its policies on data collection, use and retention will be implemented and enforced on a platform that also operates programs with significantly different policies in these areas; and how it will interact with the vetting of passengers on international flights?

4. How will commercial data sources be used? One of the most controversial elements of Secure Flight has been the possible uses of commercial data. TSA has never clearly defined two threshold issues: what it means by “commercial data;” and how it might use commercial data sources in the implementation of Secure Flight. TSA has never clearly distinguished among various possible uses of commercial data, which all have different implications.

Possible uses of commercial data sometimes described by TSA include: (1) identity verification or authentication; (2) reducing false positives by augmenting passenger records indicating a possible match with data that could help distinguish an innocent passenger from someone on a watch list; (3) reducing false negatives by augmenting all passenger records with data that could suggest a match that would otherwise have been missed; (4) identifying sleepers, which itself includes: (a) identifying false identities; and (b) identifying behaviors indicative of terrorist activity. A fifth possibility has not been discussed by TSA: using commercial data to augment watch list entries to improve their fidelity. Assuming that identity verification is part of Secure Flight, what are the consequences if an identity cannot be verified with a certain level of assurance?

It is important to note that TSA never presented the SFWG with the results of its commercial data tests. Until these test results are available and have been independently analyzed, commercial data should not be utilized in the Secure Flight program.

5W5. Which matching algorithms work best? TSA never presented the SFWG with test results showing the effectiveness of algorithms used to match passenger names to a watch list. One goal of bringing watch list matching inside the government was to ensure that the best available matching technology was used uniformly. The SFWG saw no evidence that TSA compared different products and competing solutions. As a threshold matter, TSA did not describe to the SFWG its criteria for determining how the optimal matching solution would be determined. There are obvious and probably not-so-obvious tradeoffs between false positives and false negatives, but TSA did not explain how it reconciled these concerns.

6. What is the oversight structure and policy for Secure Flight? TSA has not produced a comprehensive policy document for Secure Flight that defines oversight or governance responsibilities.

Their conclusion?

We, the SFWG were not provided adequate information about the proposed program for Secure Flight. Therefore, we are unable to make any substantive recommendations at this time. We do, however, suggest the following actions:

Congress should prohibit live testing of Secure Flight until it receives the following from the Secretary of the Department of Homeland Security.

First, a written statement of the goals of Secure Flight signed by the Secretary of DHS that only can be changed on the Secretary’s order. Accompanying documentation should include: (1) a description of the technology, policy and processes in place to ensure that the system is only used to achieve the stated goals; (2) a schematic that describes exactly what data is collected, from what entities, and how it flows through the system; (3) rules that describe who has access to the data and under what circumstances; and (4) specific procedures for destruction of the data. There should also be an assurance that someone has been appointed with sufficient independence and power to ensure that the system development and subsequent use follow the documented procedures.

Full report can be found here on the TSA web site (pdf), or here on this site.

(Thanks to Edward Hasbrouck of the Practical Nomad for the tip. See also his comprehensive rundown of recent Secure Flight news here.)

Posted by Ryan Singel at September 23, 2005 09:58 AM

Trackback Pings

TrackBack URL for this entry:
http://www.secondaryscreening.net/cgi-bin/mt-tb.cgi/237

Listed below are links to weblogs that reference Advisory Panel Report Made Public:

» Secondary Screening: Advisory Panel Report Made Public from Privacy Digest: Privacy News (Civil Rights, Encryption, Free Speech, Cryptography)
[Read More]

Tracked on September 23, 2005 01:23 PM