President's Justification of the High Performance Computer Control Threshold Does Not Fully Address National Defense Authorization Act of 1998 Requirements
Gao ID: GAO-06-754R June 30, 2006The United States controls the export of high performance computers for national security and foreign policy reasons. High performance computers have both civilian and military applications and operate at or above a defined performance threshold (which was formerly measured in millions of theoretical operations per second [MTOPS], but is now measured in Weighted TeraFlops [WT]). The U.S. export control policy currently organizes countries into "tiers," with tier 3 representing a higher level of concern related to U.S. national security interests than tiers 1 and 2. A license is required to export computers above a specific performance level to countries such as China, India, Israel, Pakistan, and Russia. Policy objectives of U.S. computer export controls are to (1) limit the acquisition of highest-end, high performance computer systems by potential adversaries and countries of proliferation concern and (2) ensure that U.S. domestic industries supporting important national security computer capabilities can compete in markets where there are limited security or proliferation risks. Over the last few years, the effectiveness of U.S. export controls in meeting these policy objectives has been challenged by market and technological changes in the computer and microprocessor industries. The National Defense Authorization Act of 1998 requires that the President provide a justification to Congress for changing the control threshold for exports of high performance computers to certain sensitive countries. The President's report must, at a minimum, (1) address the extent to which high performance computers with capabilities between the established level and the newly proposed level of performance are available from foreign countries, (2) address all potential uses of military significance to which high performance computers at the newly proposed level could be applied, and (3) assess the impact of such uses on U.S. national security interests. In February 2006, the President set a new control threshold for high performance computers and a new formula for calculating computer performance. GAO is required by law to assess the executive branch's proposed changes to the current control thresholds related to foreign availability and the national security risks of exporting high performance computers between the previous and proposed thresholds.
The President's February 2006 report did not fully address the three requirements of the National Defense Authorization Act of 1998. Therefore, the report did not present the full implications of the threshold change to Congress.
GAO-06-754R, President's Justification of the High Performance Computer Control Threshold Does Not Fully Address National Defense Authorization Act of 1998 Requirements
This is the accessible text file for GAO report number GAO-06-754R
entitled 'President's Justification of the High Performance Computer
Control Threshold Does Not Fully Address National Defense Authorization
Act of 1998 Requirements' which was released on June 30, 2006.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
June 30, 2006:
Congressional Committees:
Subject: President's Justification of the High Performance Computer
Control Threshold Does Not Fully Address National Defense Authorization
Act of 1998 Requirements:
The United States controls the export of high performance computers for
national security and foreign policy reasons. High performance
computers have both civilian and military applications and operate at
or above a defined performance threshold (which was formerly measured
in millions of theoretical operations per second [MTOPS], but is now
measured in Weighted TeraFlops [WT]).[Footnote 1] The U.S. export
control policy currently organizes countries into "tiers," with tier 3
representing a higher level of concern related to U.S. national
security interests than tiers 1 and 2. A license is required to export
computers above a specific performance level to countries such as
China, India, Israel, Pakistan, and Russia.
Policy objectives of U.S. computer export controls are to (1) limit the
acquisition of highest-end, high performance computer systems by
potential adversaries and countries of proliferation concern and (2)
ensure that U.S. domestic industries supporting important national
security computer capabilities can compete in markets where there are
limited security or proliferation risks. Over the last few years, the
effectiveness of U.S. export controls in meeting these policy
objectives has been challenged by market and technological changes in
the computer and microprocessor industries.
The National Defense Authorization Act of 1998[Footnote 2] requires
that the President provide a justification to Congress for changing the
control threshold for exports of high performance computers to certain
sensitive countries.[Footnote 3] The President's report must, at a
minimum, (1) address the extent to which high performance computers
with capabilities between the established level and the newly proposed
level of performance are available from foreign countries, (2) address
all potential uses of military significance to which high performance
computers at the newly proposed level could be applied, and (3) assess
the impact of such uses on U.S. national security interests. In
February 2006, the President set a new control threshold for high
performance computers and a new formula for calculating computer
performance.[Footnote 4]
GAO is required by law[Footnote 5] to assess the executive branch's
proposed changes to the current control thresholds related to foreign
availability and the national security risks of exporting high
performance computers between the previous and proposed thresholds. To
comply with the statutory requirement, we reviewed the three statutory
requirements for the justification, the documentation used by executive
branch officials to support the conclusions in the President's February
2006 report, and export control regulations pertaining to high
performance computers. We also interviewed officials from the
Departments of Commerce and Defense who were responsible for producing
the President's report. We addressed this report to relevant
congressional committees of jurisdiction for this mandate. We conducted
this review from February 2006 to May 2006 in accordance with generally
accepted government auditing standards.
Background:
The United States controls the export of high performance computers and
related components (e.g., microelectronics) through the Export
Administration Act of 1979 and the implementing Export Administration
Regulations.[Footnote 6] The act authorizes Commerce to require firms
to obtain licenses for the export of sensitive items that may be a
national security or foreign policy concern. The Departments of
Defense, Energy, and State assist Commerce, which administers the act,
by reviewing export applications and supporting Commerce in its reviews
of export control policy.
Beginning in 1996, the executive branch organized countries into four
computer tiers, with each tier above tier 1 representing a successively
higher level of concern related to U.S. national security interests. In
2001, tiers 1 and 2 were merged. Current U.S. export control policy
places no hardware license requirements on tier-1 or tier-2 countries,
primarily those in Africa, Asia, Central and Eastern Europe, Japan,
Latin America, and Western Europe. Exports of computers above a
specific performance level to tier-3 countries, such as China, India,
Israel, Pakistan, and Russia, require a license. Exports of high
performance computers to tier-4 countries, such as Iran, North Korea,
and Syria, are essentially prohibited. This report refers to tier-3
countries as "countries of concern."
The President has periodically changed, on the basis of technological
advances, the threshold above which export licenses are required. He
last raised the threshold in 2002 to 190,000 MTOPS. In a February 3,
2006, report to Congress, the President reported that he was replacing
the formula for calculating computer performance current at the time
with a new formula derived from industry standards that is more
accurate and easier to calculate than MTOPS. He also set a new
threshold for high performance computer exports on the basis of the new
formula.
The President's report stated that the new formula, Adjusted Peak
Performance (APP),[Footnote 7] as expressed in weighted TeraFlops, more
accurately differentiates between high-end, special-order, high
performance computers and commercial, readily available systems. It
also stated that APP places more weight on vector systems than
nonvector systems.[Footnote 8] Finally, the report stated that the 0.75
WT control level takes into consideration the widespread foreign
availability of computing capacity.
We reviewed prior justifications for changing the export control
thresholds on high performance computers and found that the changes
were not adequately justified. For example, previous President reports
failed to address all uses of military significance to which high
performance computers could be applied at the new thresholds and the
impact of such uses on national security, as required by law.
We recommended in previous GAO reports that (1) Commerce convene a
panel of experts to conduct a comprehensive assessment of possible
options to safeguard U.S. national security interests related to
computer exports, such as replacing the MTOPS performance
metric;[Footnote 9] (2) Commerce, Defense, and State assess the
national security threat and proliferation risks of U.S. exports of
high performance computers to countries of concern;[Footnote 10] and
(3) Defense determine what countermeasures are necessary, if any, to
respond to enhancements of the military or proliferation capabilities
of countries of concern derived from high performance
computing.[Footnote 11] In 2002, we recommended that Congress should
consider requiring the executive branch to fully comply with existing
statutes before the executive branch alters or eliminates the export
control threshold for high performance computers.[Footnote 12] This
would include assessing the national security and military uses of
computers at the higher threshold. In 2004, Commerce implemented the
first recommendation to convene a panel of experts, which agreed to
replace the MTOPS metric with a new APP formula. The executive branch
has not implemented our other two recommendations. A list of GAO
products related to this review is shown at the end of this report.
Summary:
The President's February 2006 report did not fully address the three
requirements of the National Defense Authorization Act of 1998.
Therefore, the report did not present the full implications of the
threshold change to Congress.
Worldwide Computer Availability Was Not Adequately Documented:
Although the President's report indicated that foreign computing
capacity below the new control level is currently widely available,
agency officials (1) did not adequately document how they established
the new export control threshold at 0.75 WT on the basis of their
assessment of worldwide availability and (2) could not document that
they verified key information used in their decision. Defense said that
a High Performance Computer Working Group, established by Commerce and
comprised of U.S. government and industry representatives, evaluated
and compared information on computer systems and performance levels
from government, industry, academia, and intelligence sources.
Officials said they also held informal meetings and discussions with
the computer industry on technology developments. We reviewed the
information the agencies collected but could not determine the basis
for selecting the new threshold compared with other possible thresholds
assessed because the agencies did not document the steps leading to
their decision.
Furthermore, officials said that they relied on a Top500 Supercomputer
List maintained by the University of Tennessee and the University of
Mannheim in Germany to help establish worldwide computer availability.
However, the information on the Top500 List is not verified by the U.S.
government; is self-reported by respondents[Footnote 13] to a
questionnaire; and may not be error-free despite their efforts to
verify it, according to the list's producers. Defense said in its
comments that the Working Group calculated the APP levels for the
Top500 List computers, compared these levels with Defense testing data
on comparable systems, and found consistent results. However, Defense
did not disclose and document how many systems from the Top500 List it
reviewed when it made these comparisons or whether these systems were
in countries of concern. Accordingly, the U.S. government determined
the worldwide availability of high performance computers on the basis
of information that might not be accurate and reliable.
Although not required by law, Commerce could have conducted a foreign
availability assessment, which is the principal mechanism identified in
the U.S. Export Administration Regulations for determining the
availability of controlled items. This assessment determines whether
items of comparable quality and quantity are available from non-U.S.
sources that would render U.S. export controls on the item
ineffective.[Footnote 14]
All Potential Military Uses of Computers Were Not Adequately Assessed:
The President's report also did not adequately assess the potential
military uses of computers with performance capabilities at the new
threshold (0.75 WT). First, the report could have disclosed, but did
not, that U.S. government officials had identified 15 high performance
computing platforms[Footnote 15] that would no longer need to be
reviewed for a license at the new control threshold.[Footnote 16]
Second, the report could have discussed, but did not, how the new
threshold would impact 29 computers that Commerce licensed with
extensive conditions for export since the executive branch last raised
the threshold in 2002 to 190,000 MTOPS. As a result of the decision to
move to the new level, all but 1 of those 29 computers would now be
eligible for export without a license and without conditions to
countries of concern, such as China, India, and Russia.[Footnote 17]
Rather than address all potential uses of military significance for
these computers, as required by law, the report provided selected
examples of computer applications, such as the design, development, and
production of weapon systems; military operations; and nuclear weapons
design and simulation. The report stated that the majority of computers
used for these purposes fall below the new and old thresholds for tier-
3 countries. It also provided some examples of national security
applications that require computer performance levels above the new
threshold. Defense officials stated that a strategic assessment of
selected applications and computing power at a given threshold would be
more useful to Congress than the assessment in the current reporting
requirement, which they said is too narrowly focused. However, the
report did not provide this assessment, either.
National Security Impact of Militarily Significant Computer Uses Was
Not Assessed:
Since the President's report did not adequately assess the potential
military uses of computers at the proposed new threshold of 0.75 WT, it
did not assess the impact that militarily significant uses of those
computers would have on U.S. national security. Instead, the report
discussed how the new threshold would allow the United States to
maintain a technological lead by controlling the less widely available,
most advanced computer systems best suited for innovative national
security applications above the new threshold. In addition, Defense
officials stated that assessing the national security impact on
computers between the old and new thresholds would be very time-
consuming and resource-intensive.
Agency Comments and Our Evaluation:
We received written comments on a draft of this report from Commerce
and Defense, which are reprinted in enclosures I and II. Commerce and
Defense disagreed with our finding that the executive branch did not
have a rigorous process to establish the new computer control
threshold. The agencies stated that an interagency group compared
information on computer systems and performance levels from government,
industry, academia, and intelligence sources, and discussed the new
threshold with foreign governments. The agencies also said that the
U.S. government verified information on the Top500 List upon which they
relied to evaluate state-of-the-art systems and to help calculate the
APP levels for high performance computers. Accordingly, Commerce stated
there is no basis to assert that worldwide availability was determined
on information that might not be accurate and reliable.
Defense also disagreed with our finding that the President's report did
not adequately assess the potential military uses of computers with
performance capabilities at the new threshold or the impact such
militarily significant uses would have on U.S. national security. It
stated that Defense did not provide an exhaustive list of all potential
military applications because the number and diversity of military
applications and the rate of change in high performance computing make
comprehensive analysis of all potential applications, their national
security impact, and appropriate responses impractical. In addition,
Defense stated that we did not recognize the pace of technology
developments in computing and stated that systems have an average
lifespan of 3 to 6 years. Thus, according to Defense, many of the
computers exported during that time are or are becoming obsolete.
In response, we added some additional information to this report to
more fully describe the process and sources of information that
Commerce used to establish the new export control threshold. Although
the interagency group collected and analyzed a large amount of raw data
on computer performance levels, the documentation that the agencies
provided did not demonstrate how they met the requirements of the law
or provide a step-by-step approach that an outside reviewer could
verify. For example, we could not confirm that the U.S. government
verified information on the Top500 List. Commerce stated that agencies
verified the list by comparing the APP performance level of computers
on the Top500 List with benchmark data on comparable U.S. systems.
However, Commerce did not provide documentation on how many Top500 List
computers Defense reviewed or whether the systems assessed included
those manufactured by countries of concern. Moreover, Defense officials
said in interviews that they did not adequately document how they
established the new export control threshold at 0.75 WT on the basis of
their assessment of worldwide availability.
We do recognize the rapid pace of technological change in the computing
industry. However, it is important to note that because certain
computer systems are no longer as useful to the U.S. government as
newer systems does not mean that these computers would not have some
militarily significant use for a country of concern.
Defense's assertion that it was impractical to provide an exhaustive
list of all potential military applications to meet the requirements of
the law is not noted in the President's report to Congress nor did the
report indicate that the executive branch sought legislative relief
from this mandated requirement.
Scope and Methodology:
To address the requirements of the National Defense Authorization Act
of 1998, we reviewed the statutory requirements for the justification,
the documentation that executive branch officials said they used to
support the conclusions presented in the President's report, and the
export control regulations pertaining to high performance computers. We
also interviewed officials from the Departments of Commerce and Defense
who were responsible for producing the report.
We are sending this report to interested congressional committees and
to the Secretaries of Commerce and Defense. We will also make copies
available to other interested parties on request. In addition, the
report will be available at no charge on the GAO Web site at
[Hyperlink, http://www.gao.gov].
Please contact me at (202) 512-8979 if you or your staff have any
questions concerning this report. Contact points for our Offices of
Congressional Relations and Public Affairs may be found on the last
page of this report. Key contributors to this report include Stephen M.
Lord, Jeffrey D. Phillips, Ian Ferguson, Lynn Cothern, Hai Tran, and
Mark Speight.
Signed by:
Joseph A. Christoff:
Director, International Affairs and Trade:
List of Congressional Committees:
The Honorable John Warner:
Chairman:
The Honorable Carl Levin:
Ranking Minority Member:
Committee on Armed Services:
United States Senate:
The Honorable Richard C. Shelby:
Chairman:
The Honorable Paul S. Sarbanes:
Ranking Minority Member:
Committee on Banking, Housing, and Urban Affairs:
United States Senate:
The Honorable Duncan L. Hunter:
Chairman:
The Honorable Ike Skelton:
Ranking Minority Member:
Committee on Armed Services:
House of Representatives:
The Honorable Henry J. Hyde:
Chairman:
The Honorable Tom Lantos:
Ranking Minority Member:
Committee on International Relations:
House of Representatives:
Enclosure I: Comments from the Department of Commerce:
United States Department Of Commerce:
Bureau of Industry and Security:
Washington, D.C. 20230:
June 9, 2006:
Joseph A. Christoff:
Director, International Affairs and Trade:
United States Government Accountability Office:
Washington, DC 20548:
Dear Mr. Christoff:
Thank you for the opportunity to review the Government Accountability
Office (GAO) draft numbered correspondence entitled President's
Justification of the High Performance Computer Control Threshold Does
Not Fully Address NDA Act Requirements (GAO-06-754R). The draft
correspondence suggests that the President's February 2006 report on
revisions to the high performance computer (HPC) controls did not fully
address the foreign availability portion of the National Defense
Authorization Act requirements.
The Administration conducted a thorough review of U.S. export controls
on high performance computers (HPCs) in the two years leading up to the
President's February 3, 2006 decision to revise our HPC export control
policy to Tier 3 countries. This review involved extensive discussions
among all relevant agencies, including the Departments of Commerce,
State, Defense, and Energy, on the national security impact of
modifying the existing controls, identifying a new formula (Adjusted
Peak Performance (APP)) to replace the Composite Theoretical
Performance (CTP) formula, and setting the new licensing threshold at
0.75 WT (Weighted TeraFLOPS). The Administration studied the advances
in microprocessor technology and computer architectures, examined the
widespread foreign availability of computing capacity, and evaluated
the competitiveness of the U.S. computer industry in the global market
place.
This review was informed by the "Top 500 Supercomputers List" - a list
well-known to the government experts as a reliable source of data on
the world-wide availability of computing capacity. The Top 500 List is
a comprehensive and reliable resource for evaluating the state-of-the
art in high performance computing. This global list is used by
academic, government, and industry experts worldwide for tracking and
identifying trends in high performance computing. It should be noted
that 305 of the 500 systems in the November Top 500 List are located in
the United States and a number of them are owned and operated by the
U.S. Government. Furthermore, a company or individual who wants to
enter his machine in the Top 500 List must provide the results of his
Linpack benchmark test and submit the results to random verification by
the organizer. The Top 500 List is studied and watched by people around
the world. Overstating HPC performance would certainly have negative
repercussions on a manufacturer/exporter's credibility and sales.
Moreover, as noted in the Department of Defense's comments on the draft
correspondence, agencies compared the APP performance levels of
computers on the Top 500 List with the Defense benchmark data on
comparable systems and found consistent results. Thus, information on
the Top 500 List was verified by the U.S. government.
In addition, the techniques employed in achieving the stated
performance are repeated by different people around the world. Many of
the tools are available from open sources and others have reported
similar results. There is no reason to suspect that foreign vendors
have been exaggerating their results. Also, Western market research
groups and consultants, such as IDC and the Asian Technology
Information Program, have visited these foreign sites and reported on
them. Further evidence of the reliability of the Top 500 is the support
for the APP formula and control level by the Government of Japan and
our Wassenaar Arrangement regime partners based on the data in the Top
500 List.
Conducting a formal foreign availability assessment under part 768 of
the Export Administration Regulations would have resulted in a very
similar interagency review process. Such an assessment would have also
relied on the Top 500 List because it represents the most comprehensive
and transparent documentation of HPC performance worldwide.
The most recent Top 500 List underscores the widespread availability of
computer clusters that have increased in both performance and
applications. In the most recent Top 500 List, released in November
2005, 360 systems are clusters.
In light of the above, the second full paragraph on page 5 of the draft
correspondence should be significantly revised. Agency officials did
conduct a rigorous, well-founded review to determine worldwide
availability. Agency officials did verify entries on the Top 500 List.
Thus, there is no basis for asserting that worldwide availability was
determined on information that might not be accurate and reliable.
Two other points are worth noting. On page 5, the draft correspondence
states that "U.S. Government officials had identified 15 types of high
performance computers that would no longer need to be reviewed for a
license at the new control threshold." While a license will not be
required for HPCs in those configurations, licenses will continue to be
required for larger configurations. The computer platforms are not
fully decontrolled but only less capable versions.
Finally, the licensing data illustrates the rapid advance of
technology. In addition to the one HPC approved prior to December 31,
2005 that would continue to require a license for export to Tier 3
counties, Commerce has approved two HPC license applications, each with
WT value over 0.75. Therefore, there are now three HPC licenses with WT
values over 0.75 to Tier 3 countries. As previous adjustments to the
HPC licensing threshold, we expect a slowdown in license applications
in the short term but the number of license applications will rise
again as technology advances. Industry has already announced
performance improvements with multi-core processor technology in the
coming years.
Once again, I appreciate the opportunity to comment on the draft
correspondence. Our point of contact on this issue is Bernard Kritzer,
Director of the Office of National Security Controls and Technology
Transfer. Mr. Kritzer can be reached at (202) 482-5491.
Sincerely,
Signed by:
Matthew S. Borman:
Deputy Assistant Secretary:
The following are GAO's comments on the Department of Commerce letter
dated June 9, 2006.
GAO Comments:
1. We added additional information to this report to more fully
describe the administration's review of U.S. export controls on high
performance computers and sources of information that Commerce used to
establish the new export control threshold and the role of the High
Performance Computer Working Group.
2. Commerce stated that a company representative or individual who
wants to enter his or her machine on the Top500 Supercomputer List must
provide the results of testing and submit the results to random
verification by the list's organizer. However, Commerce did not provide
any documentation on how many or which systems the list's organizer
randomly verified. Moreover, the list's producers acknowledged that
information on the list is self-reported by respondents to a
questionnaire and may not be error-free. Commerce also stated in its
comments that agencies compared the Adjusted Peak Performance (APP)
level of computers on the Top500 List with Department of Defense
benchmark data on comparable systems and found comparable results.
However, Commerce did not state or provide documentation on how many or
which Top500 List computers Defense reviewed that were comparable to
U.S. government systems. This would be important information to
determine how many of the 500 systems the U.S. government reviewed and
whether they included systems in or manufactured by countries of
concern. Finally, Commerce stated that the Top500 List was verified by
the U.S. government. On the basis of the documentation that the
agencies provided, we cannot confirm that the U.S. government verified
information on the Top500 List. In our original audit work, Defense and
Commerce officials stated that they did not perform an independent
analysis of computers produced by tier-3 countries, but that they
relied on other sources of information to estimate the performance of
such computers.
3. We disagree that conducting a formal foreign availability assessment
would have resulted in a similar interagency review process. The
President's report provided only anecdotal examples of the availability
of high performance computers in countries of concern, such as China,
India, and Russia, even though the report asserted that the new control
threshold was based on the widespread foreign availability of computing
capacity. A foreign availability assessment is the principal mechanism
identified in the U.S. Export Administration Regulations for
determining the availability of controlled items. This assessment would
determine whether items of comparable quality and quantity are
available from non-U.S. sources, thus rendering U.S. export controls on
the item ineffective.
4. We disagree with Commerce's statement that the President's report
was based on a rigorous, well-founded review to set the new control
threshold and determine worldwide availability of high performance
computers. Although the Working Group collected a large amount of
information and raw data on computer performance levels, we found in
interviews with Defense officials and our review of their data that
agencies did not document their methodology or the steps leading to the
new threshold of 0.75 Weighted TeraFlops (WT). The documentation that
the agencies provided did not demonstrate a step-by-step approach that
an outside reviewer could follow to verify how they reached their
threshold decision.
5. We acknowledge these technical comments and modified our report to
address both points.
Enclosure II: Comments from the Department of Defense:
Defense Technology Security Administration:
2900 Defense Pentagon:
Washington, DC 20301-2900:
Mr. Joseph A. Christoff:
Director, International Affairs and Trade:
U.S. Government Accountability Office:
441 G Street, N.W.
Washington, D.C. 20548:
May 31 2006:
Dear Mr. Christoff:
This is the Department of Defense (DoD) response to the GAO draft
report, "President's Justification of the High Performance Computer
Control Threshold Does Not Fully Address NDA Act Requirements," dated
May 19, 2006, (GAO Code 320413/GAO-06-754R). Thank you for the
opportunity to review the draft report. DoD comments and clarifications
are provided in detail below.
The draft report states that the interagency (Departments of Commerce,
Defense, State, and Energy) did not have a rigorous methodology or
process to determine foreign availability and establish a computer
control threshold. In fact, the High Performance Computer (HPC) Working
Group, established by the Department of, Commerce and comprised of
interagency and industry representatives, derived the new computer
control metric Adjusted Peak Performance (APP) and the 0.75 Weighted
TeraFLOPS (WT) threshold based on a methodology that evaluated and
compared information on computer systems and performance levels from
government, industry, academia, and intelligence sources.
The Top 500 List represents the types of HPCs being built and used
world-wide and was used by the HPC Working Group to evaluate the state-
of-the-art in general-purpose computing and to confirm the qualitative
difference between vector and scalar systems. The APP was calculated
for the Top 500 List and the resulting rank ordering of systems
reviewed by both government and industry. The interagency compared the
APP performance levels of computers on this list with the results of
DoD benchmark data on comparable systems and found consistent results.
The interagency also looked at how well DoD Challenge Project software
programs ran on comparable systems and again found consistency in
performance ranking between the Top 500 List and DoD systems tested
using APP.
In setting the threshold, the HPC Working Group considered a number of
factors, including the ability of other countries to assemble HPC
systems and the rate of advance of HPCs. Additionally, the interagency
consulted with the Government of Japan, as the only other producer of
HPCs and as part of the US-Japan HPC Regime, to allow the Government of
Japan to consult with their industry to test APP and ensure that 0.75
WT was a mutually accurate control threshold.
Commercial developments in commodity computing and consumer electronics
are and will continue to rapidly encroach on licensing thresholds
established to protect proprietary, scientific computing platforms.
Small systems produced in large quantities, such as blade servers, and
licensed by US and Japanese industries for worldwide production (e.g.,
in Taiwan, China, Singapore, Malaysia, and the Philippines), will soon
exceed the licensing level of 190,000 MTOPS established in March 2002.
The claim that under the new threshold only one computer of 29 licensed
between March 2002 and January 2006 would need a license does not
recognize the pace of technology developments in computing. Systems
have an average lifespan of 3-6 years. As those systems become
obsolete, they are replaced with newer, faster, and more efficient
systems. Many of the computers exported during that time frame are or
are becoming obsolete. In addition, powerful commodity clustered
systems are being built from widely available commodity components and
interconnects.
The draft report states that the interagency did not adequately assess
the potential military uses of computers with performance capabilities
at the new threshold (0.75 WT), nor did it assess the impact that
militarily significant uses of computers between the old and new
thresholds would have on US national security. DoD did not provide an
exhaustive list of all potential military applications because the
number and diversity of military applications and the rate of change in
high performance computing make comprehensive analysis of all potential
applications, their impact, and appropriate responses impractical in
evaluating the national security impact of changes in computer control
levels. Rather, the HPC Working Group evaluated the effectiveness of
proposed controls by looking at the 94 HPC systems in use in 2004 by
the US Government for military and nuclear related research and
development. This approach allowed the HPC Working Group to focus on
systems of greatest utility to the HPC community. As noted in meetings
with the GAO, national security related work is conducted across all
computing performance levels - from desktop workstations available from
multiple sources around the world, to high-end, proprietary HPCs
available only in the United States and Japan. Export controls on
computers cannot prevent a potential adversary from pursuing national
security related work; controls can only limit access to the most
efficient and reliable computers being used by the USG for
computationally intensive military and intelligence related work. DoD
invests heavily in high performance computing (hardware and software)
to ensure that the research and development community has the most
reliable and effective computing tools for development and testing of
future combat and warfare systems.
Finally, DoD actively pursues measures to protect militarily critical
software through the Software Protection Initiative (SPI). The SPI and
focused and effective controls on proprietary computer hardware and
technology that protect and promote a robust industrial computing base
are the most effective countermeasures to ensure US technological lead
and protect the technological edge of the US warfighter.
Sincerely,
Signed by:
Beth M. McCormick:
Director (Acting),
Defense Technology Security Administration:
Enclosure:
DoD Technical and Editorial Comments:
The following are GAO's comments on the Department of Defense letter
dated May 31, 2006.
GAO Comments:
1. We added additional information to our report to more fully describe
the process and sources of information that Commerce used to establish
the new export control threshold, and the role of the High Performance
Computer Working Group. This group was not cited in the President's
report. Although the Working Group collected and analyzed a large
amount of information and raw data on computer performance levels,
Defense officials said in interviews that they did not document their
methodology or how they set the new threshold at the 0.75 WT level. The
documentation that the agencies provided did not demonstrate a step-by-
step approach that an outside reviewer could follow to verify how they
reached their threshold decision.
2. Defense stated in its comments that agencies compared the APP level
of computers on the Top500 List with Defense benchmark data on
comparable systems and found consistent results. However, Defense did
not state or provide documentation on how many or which Top500 List
computers agencies reviewed that were comparable to U.S. government
systems. This would be important information to determine how many of
the 500 systems the U.S. government reviewed and whether they included
systems in or manufactured by countries of concern.
3. We recognize the rapid pace of technological change in the computing
industry. However, because certain computer systems are no longer as
useful to the U.S. government as newer systems does not mean that these
computers would not have some militarily significant use for a country
of concern. Moreover, the President's report does not identify the
estimated numbers of computers available at a specific computing level
in countries of concern. Thus, the report does not adequately assess
the potential military uses or impact on U.S. national security that
availability of these computer systems to countries of concern without
a license would have.
4. Defense stated that it did not provide an exhaustive list of all
potential military applications because the number and diversity of
military applications and the rate of change in high performance
computing make comprehensive analysis of all potential applications,
national security impacts, and appropriate responses impractical.
Defense officials said that to generate a list of applications to be
used on certain levels of computers would require reviewing and
analyzing an extensive amount of information. However, the President's
report did not state the Defense view that the requirement is
impractical, nor did the agencies indicate that they appealed for
legislative relief.
5. The National Defense Authorization Act of 1998 requires the agencies
to assess the potential military uses of computers at the new threshold
and, therefore, the impact that changes in the control threshold will
have on U.S. national security. Merely stating that Defense conducts
national security-related work on computers at all performance levels
does not respond to the requirement of the act to assess the national
security uses that computers at the new performance threshold can have.
Thus, we believe it would be useful for the executive branch to conduct
this assessment and provide the results to Congress.
Related GAO Products:
Export Controls: More Thorough Analysis Needed to Justify Changes in
High Performance Computer Controls. GAO-02-892. Washington, D.C.:
August 2, 2002.
Export Controls: Inadequate Justification for Relaxation of Computer
Controls Demonstrates Need for Comprehensive Study. GAO-01-534T.
Washington, D.C.: March 15, 2001.
Export Controls: System for Controlling Exports of High Performance
Computing Is Ineffective. GAO-01-10. Washington, D.C.: December 18,
2000.
Export Controls: Challenges and Changes for Controls on Computer
Exports. GAO/T-NSIAD-00-187. Washington, D.C.: May 26, 2000.
Export Controls: National Security Risks and Revisions to Controls on
Computers. GAO/T-NSIAD-00-104. Washington, D.C.: February 28, 2000.
Export Controls: Statutory Reporting Requirements for Computers Not
Fully Addressed. GAO/NSIAD-00-45. Washington, D.C.: November 5, 1999.
Export Controls: Implementation of the 1998 Legislative Mandate for
High Performance Computers. GAO/T-NSIAD-00-53. Washington, D.C.:
October 28, 1999.
Export Controls: Information on the Decision to Revise High Performance
Computer Controls. GAO/NSIAD-98-196. Washington, D.C.: September 16,
1998.
Export Controls: Changes in Controls Applied to the Export of High
Performance Computers. GAO/T-NSIAD-98-250. Washington, D.C.: September
16, 1998.
Export Controls: National Security Issues and Foreign Availability for
High Performance Computer Exports. GAO/NSIAD-98-200. Washington, D.C.:
September 16, 1998.
(320413):
FOOTNOTES
[1] A TeraFlop is a measure of the mathematical computing power of a
machine in floating point operations per second. A floating point
operation is meant to be an elementary operation, such as a sum,
subtraction, multiplication, or division. This measure is used to
express the speed of a processor.
[2] Public Law 105-85, sec. 1211(d), 111 stat. 1933.
[3] The Departments of Defense, Commerce, State, and Energy prepare the
President's report under the coordination of the National Security
Council.
[4] The White House, Summary of Findings with Respect to Criteria Set
Forth in Subsections 1211(d) and (e) of the National Defense
Authorization Act for Fiscal Year 1998 (Washington, D.C.: Feb. 3,
2006).
[5] Public Law 106-554, app. B, sec. 314, 114 stat. 2763A-123.
[6] 50 U.S.C. app. 2401 et seq., and 15 C.F.R., sec. 730. The
authorities provided by this legislation expired on August 20, 2001.
However, acting pursuant to the International Emergency Economic Powers
Act (codified at 50 U.S.C. 1702), the President extended application of
the act and the implementing regulation by Executive Order 13222 (66
Fed. Reg. 44025).
[7] The metric addresses our past recommendation to assess ways to
address the shortcomings of high performance computer export controls
based on MTOPS. See our report titled Export Controls: System for
Controlling Exports of High Performance Computing Is Ineffective, GAO-
01-10 (Washington, D.C.: Dec. 18, 2000). However, we did not assess
whether APP is the most appropriate alternative to MTOPS.
[8] A vector system is a computer with built-in instructions that
performs multiple calculations on vectors (one-dimensional arrays)
simultaneously.
[9] GAO-01-10.
[10] GAO, Export Controls: More Thorough Analysis Needed to Justify
Changes in High Performance Computer Controls, GAO-02-892 (Washington,
D.C.: Aug. 2, 2002); and Export Controls: Information on the Decision
to Revise High Performance Computer Controls, GAO/NSIAD-98-196
(Washington, D.C.: Sept. 16, 1998).
[11] GAO-01-10.
[12] GAO-02-892.
[13] Respondents include high performance computer experts,
computational scientists, manufacturers, and the Internet community,
according to the Top500 List's producers.
[14] 15 C.F.R. 768.
[15] Computing platforms can operate at various performance levels
based on the number of processors available.
[16] Licenses will continue to be required for some of these high
performance computers operating with a larger configuration of
processors, according to Commerce.
[17] Since December 31, 2005, Commerce stated that it approved licenses
to export 2 additional computers above the new threshold of 0.75 WT to
countries of concern, such as China, India, and Russia.
GAO's Mission:
The Government Accountability Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office
441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director,
NelliganJ@gao.gov
(202) 512-4800
U.S. Government Accountability Office,
441 G Street NW, Room 7149
Washington, D.C. 20548:
