Cybersecurity Canon Candidate Book Review: The Shadow Factory: The Ultra-Secret NSA from 9/11 to the Eavesdropping on America

We modeled the Cybersecurity Canon after the Baseball Hall of Fame and the Rock & Roll Hall of Fame, except for cybersecurity books. We have more than 25 books on the initial candidate list, but we are soliciting help from the cybersecurity community to increase the number. Please write a review and nominate your favorite.

The Cybersecurity Canon is a real thing for our community. We have designed it so that you can directly participate in the process. Please do so!

Executive Summary

Like the attack on Pearl Harbor a generation before, the attacks on 9/11/2001 were seen as a turning point for U.S. security and policy, and the National Security Agency (NSA) was deeply impacted by this transition. Before 9/11, congress viewed the NSA as a relic of the cold war and sought to capitalize on the “peace dividend” with deep budget and staffing cuts. That attitude changed radically after the attacks.

With new authority and massive budget increases, the NSA was asked to ramp up its programs by whatever means necessary. Some of these programs straddled legal and ethical lines, leading to robust debate over constitutional authority, executive power, and what the U.S. stands for as a country. Others can be viewed in retrospect as high-budget science projects that led to massive payments to federal systems integrators, but did little to protect the US from further attacks.

The Shadow Factory is an excellent and detailed analysis of the evolution of the NSA during the post-9/11 timeframe (2001-2009). The author, James Bamford, is well-qualified to document this progression, as he has followed and published several other books about the NSA since the 1980s.

While The Shadow Factory is well-written and extremely detailed, it does not provide a thorough examination of the NSA’s offensive cyber operations, research, or intelligence and therefore does not qualify as a candidate for the Cybersecurity Canon.

Why then should any cybersecurity professional read this book? The 9/11 attacks occurred in part because of intelligence failures, uncoordinated organizations, collaboration challenges, and disconnected technologies — issues to which all cybersecurity professionals can relate. After the attacks, the NSA was tasked with bolstering the nation’s ability to prevent, detect, and respond to threats in a timely and efficient manner. This mission is entirely consistent with cybersecurity objectives.

Once I realized that this book was not about NSA cybersecurity operations, I immediately changed my expectations as a reader. Rather than look for InfoSec tidbits, I read Bamford’s text and interpreted the story through a fundamental question: What can a cybersecurity professional learn from the post-9/11 experiences at the NSA?  My book review attempts to answer this question.

Review

James Bamford’s The Shadow Factory chronicles the U.S. National Security Agency (NSA) from the origins of the terrorist attacks of 9/11/2001 and then follows the impact of these events on the agency throughout most of the remainder of the decade. The book is divided into five sub-sections: Book 1, Book 2, Book 3, Book 4, and Book 5.

Book 1: Attack. This section tracks some of the individuals who ultimately participated in the terror attack, following their radicalization, indoctrination, training, and roles. In 90 pages, Bamford documents how the participants entered the United States and then follows their mundane lives as they planned, collaborated, and eventually conducted the attacks. Book 1 also carefully deconstructs how the terrorists flew under the radar of U.S. intelligence and law enforcement agencies.  As has been documented elsewhere, Bamford reveals FBI, CIA, and NSA intelligence, providing an assortment of clues and missed opportunities that should have triggered alarms somewhere. For example, a member of the group attended a planning meeting in Laurel, MD, less than a week before 9/11 and then proceeded to drive to Newark, NJ, after the meeting was completed. Driving at over 90 miles per hour, he was stopped by a Maryland state trooper and asked to produce his driver license. Even though the license contained a phony address, the terrorist was given a speeding ticket and freed to go.

Book 1 concludes by describing activities at NSA and the Pentagon as the attacks occurred.  Before the attack, NSA director Michael Hayden had spent a good deal of his time trying to protect his agency from deep budget cuts driven by the end of the cold war and subsequent “peace dividend”. This all changed on Sept. 11 however. The rest of The Shadow Factory examines the extent of these changes.

Book 2: Targets. In the chapters of Book 2, Bamford looks at the immediate aftermath of 9/11 and its impact on the NSA.  It was clear from September 12, 2001, that the NSA would shift its focus to detecting terror plots and terrorists and then sharing this information with other intelligence and law enforcement agencies. Indeed, there was immense pressure (led by the administration) to do so as quickly as possible. NSA budgets and personnel were sharply increased. As an example, 40 percent of the NSA workforce in 2008 was hired after the 9/11 attacks. Still, Hayden and Co. remained challenged by obstacles like the Foreign Intelligence Surveillance Act of 1978 (FISA), a lack of Arabic-speaking analysts, and a growing need to support military actions in Afghanistan and later Iraq. Bamford details the chaos and consequences associated with this abrupt mission change.

Book 3: Cooperation.  Faced with immense challenges, the NSA knew it needed a lot of help. In Book 3, Bamford records the agency’s outreach to achieve its mission. Hayden immediately turned to friendly intelligence agencies, leaning heavily on the ‘’Five Eyes” ( Australia, Canada, New Zealand, the UK, and the U.S.). Of these, the British government’s communications headquarters (GSHQ) proved especially cooperative. For example, Bamford describes how GCHQ and others eavesdropped on UN members to gauge how they would vote in a resolution about a military response to Iraq’s weapons programs. The U.S. then used this information to persuade (bribe?) the doubting countries to support its Iraq war efforts.

Book 3 also looked at the participation of non-state actors like telecommunications companies, internet service providers, surveillance technology vendors, and even government integrators (aka “Beltway Bandits”). Bamford points out that in 2001, NSA had 55 external contracts with 144 contractors.  In 2005, this ballooned to 7197 contracts with 4,388 contractors. These partners were called upon to install surveillance equipment, tap into networks, or provide offshore support for NSA programs when necessary.

Book 4: Discovery. This book looks at the eventual backlash on NSA programs within and outside of the government. Bamford reminds readers of the shameful attempt to get Attorney General John Ashcroft to reauthorize President Bush’s domestic surveillance program while recovering from illness in a hospital bed. Deputy AG James Comey intercepted White House advisors as they arrived at the hospital, prompting Ashcroft to refuse the request. Book 4 also documents the back-and-forth negotiations between The New York Times and the government regarding a story revealing the illegal domestic surveillance program. It then looks at the government’s reaction to the news. Despite some early anger and embarrassment about the exposure, Bamford explains that little changed as a result, even with the change of administrations in 2008.

Book 5: Future. Bamford’s book concludes around 2007, well before Edward Snowden described massive NSA surveillance programs in 2013. Yet, Bamford’s book really telegraphs Snowden’s revelations. Book 5 describes massive new NSA facilities in Georgia and Colorado in support of efforts like the Novel Intelligence from Massive Data (NIMD), a surveillance database of over 48 petabytes. The NSA needed unprecedented computing power for this type of data analysis, recruiting companies like Cray and IBM to build teraflop and later petaflop computing engines. In the end, Bamford paints an alarming and somewhat tragic picture of the future of the NSA. He outlines ambitious data analytics programs like Trailblazer that garnered a budget of $280 million in 2002 and was then fraught with years of cost overruns. Despite this investment, Trailblazer was overly complex, never worked, and was ultimately disbanded.  Through this and other examples, Bamford seems to be hinting that the NSA may have reached a point at which its haystack was too big to find any needles in a timely manner.

Readers interested in national security, intelligence gathering, and the NSA will find The Shadow Factory to be a captivating read. Much to my disappointment however, the book only dabbles in NSA cyber operations, so it really doesn’t qualify as a “must read” for all cybersecurity professionals, per the Cybersecurity Canon definition. As a reviewer, I recognized the lack of cybersecurity focus early, in Book 2. Henceforth, I tried to analyze The Shadow Factory through a cybersecurity lens. In other words, my goal was to take Bamford’s description of post-9/11 NSA actions and then interpret them as practical advice for the cybersecurity community. Here are few examples of lessons learned:

1. Work as a team with other organizations. We now know that the FBI, CIA, NSA, and others had breadcrumb clues as to the whereabouts and histories of the terrorist cabal well before 9/11, but organizational and political differences precluded cooperation that could have detected and prevented the attacks. To overcome similar problems, cybersecurity professionals must build strong relationships with business professionals, industry groups, and IT operations teams to create strong communications channels and cooperative processes for intelligence sharing, risk management, and incident detection/response.
2. Document and test incident-response processes. Few anticipated an attack of the size and scope of 9/11 before 2001 (reviewers note: they should have). Once the attacks happened however, federal agencies stumbled through a long process of figuring out how to react. Security professionals can learn from this example through comprehensive threat modeling and planning for a worst-case scenario. Incident response plans should be formalized, documented, and tested, defining the roles and responsibilities of all constituencies, including business executives, HR, legal teams, public relations, IT, and cybersecurity.
3. Understand your limitations. Bamford seems to believe that the NSA wasted lots of time and enormous amounts of money on pie-in-the-sky programs and technologies. While cybersecurity professionals may be similarly tempted to engage in homegrown software development projects in areas like data science and machine learning, CISOs should approach these initiatives carefully, assessing whether their organization has the time, resources, and skills necessary for success.

Conclusion

While educational and enthralling at times, The Shadow Factory by James Bamford does not detail cyber operations at the NSA and therefore cannot be recommended as a candidate for the Cybersecurity Canon. At the same time, The Shadow Factory is an interesting and informative book for cybersecurity professionals interested in national security, intelligence collection, and the NSA. It also does a great job of following the individuals responsible for and the actions that led to the attacks on 9/11/2001. While these attacks remain a national nightmare, readers may find it interesting to understand what happened, how it happened, and how the NSA responded in the years that followed.