Amerithrax review: Lessons for future investigations

When the National Academy of Sciences issued its review of the FBI anthrax investigation earlier this year, the press fixated primarily on one point: The report found no conclusive evidence that Bruce Ivins, the Army scientist the government contends was responsible for a series of anthrax-laced letters mailed in 2001, produced them. Far less attention was devoted to the academy’s comments on the FBI’s problems with methodology — particularly the thoroughness of its research and its use of new technologies — and organization, especially at the start of the probe, when there was no structure for directing its scientific efforts. If the FBI fails to address these problems, it will remain ill-equipped to produce evidence for future bio-attack cases that stand up in court. Further, it will be unable to provide a president with sufficient information to support a possible retaliatory decision in the case of a state-sponsored bio-attack.

Perhaps the most astonishing result of the academy’s analysis involves the scope of the FBI’s findings in regard to the five anthrax letters at the heart of the case. After years of investigation, the bureau answered only a small number of the questions that it pursued: when the anthrax was produced, whether the material in all the letters was the same, and whether it had been weaponized. But other key questions relating to how and where the material was produced remained unanswered, making a determination of the material’s origin and the perpetrator’s identity problematic.

Early in the investigation, the FBI determined that the anthrax in the envelopes was of the virulent Ames strain and was produced in two separate batches with “differences in spore concentrations, color, contaminants, texture, growth media remnants and observed debris.” The presence of silicon in the material was initially deemed indicative of weaponization, as silicon dioxide has been used in past state bioweapons programs to enhance dispersion. Further analysis of the anthrax, however, indicated that the silicon was located within the spores, as the product of the spores’ natural growth process, rather than on their surface, where such dispersant material would normally reside. Carbon dating also determined that the spores were produced between 1998 and 2001, providing a possible chronological link with Ivins’s flask, labeled RMR-1029, which had been stored at US Army Medical Research Institute of Infectious Diseases (USAMRIID) since 1997. The FBI theorized that the flask was the source of the anthrax used in the 2001 attacks.

But genetic analysis was not able to ascertain the origin of the material in the five letters, which were sent to New York and Florida media outlets and to US senators Patrick Leahy and Thomas Daschle in Washington, DC. It also couldn’t identify the production methods used. Indeed, the academy’s analysis concluded that the RMR-1029 flask was not the immediate source of the letter material because it presented characteristics not shared with the letters. For example, unlike the anthrax spores in the letters, the RMR-1029 spores did not contain silicon in the coat. In addition, the RMR-1029 sample tested positive for the purification compounds meglumine and diatrizoate, which the letter samples did not contain, possibly indicating that a different purification method was used for the letter material. Therefore, the academy’s report concludes, if the letter material was indeed derived from the RMR-1029, several additional preparation steps involving cultivation, purification, and drying would have been necessary.

But the FBI investigation was not able to clarify what methods might have been used to grow, purify, concentrate and dry the anthrax sent in the letters. Given these uncertainties, the FBI was unable to determine the amount of time needed to produce the powdered anthrax, the type of skills and the equipment required to make it, the quality of the starting material, and the amount of liquid anthrax required to produce the five grams or so of dried anthrax in the letters.

Organizational issues. Probably the most important organizational flaw in the FBI probe was the lack of a coordinating structure at its outset. Because the FBI did not have the required scientific expertise to conduct an investigation of such breadth, 29 government, university, and private high-containment laboratories were commissioned to assist them. Several expert panels were also formed to review aspects of the scientific work, while additional outside expertise was sought from several government agencies. But during the first two years of the FBI investigation, no organizational structure within the bureau was responsible for overall oversight and coordination of the work performed by all these actors — or for advising the FBI on the specific expertise required and the methodology choices available.

This changed in 2003, when the FBI created the Chemical, Biological, Radiological and Nuclear Sciences Unit, also known as CBSU. It seems, however, that the creation of CBSU did not solve all of the problems. For instance, the academy notes that near the end of its investigation, the FBI called on the expertise of statisticians; their involvement would have been useful throughout the process to provide advice on different scientific aspects of the case, from experimental design to data interpretation. Absent such assistance, and because it reached uncertain or ambiguous findings, the investigation naturally fell prey to controversy.

The 29 laboratories worked for the FBI in a compartmentalized fashion, unaware of what other labs were doing. This organizational choice was probably due to a dilemma the FBI faced: The scientists capable of assisting in the investigation could have also been potential suspects. Allowing them to review research conducted by other scientists could have tipped the perpetrator, or perpetrators, off. Still, compartmentalization runs counter to the scientific process, which relies on open exchange and collective interpretation of data.

Methodology issues. The academy’s report points to a number of methodology issues bearing on the thoroughness of the FBI’s research, its technical decision-making process, and its use of new technologies.

To help its investigation, the FBI created a repository of Ames anthrax strains. The purpose of the repository was to compare the anthrax in the letters to specimens obtained from various sources, in hopes of identifying the origin of the letter material. The FBI collected 1,070 specimens from 15 US organizations and three organizations in Great Britain, Canada, and Sweden, each having been requested to provide two samples of all the Ames strains they had in their collections. One sample was used for analysis, the other was archived. In addition, the US intelligence community provided samples from an unspecified foreign location (possibly Afghanistan, where Al Qaeda was suspected of developing anthrax).

The academy questioned how the FBI’s repository was established and populated. Because the Ames strain was widely used and exchanged among laboratories in the United States and overseas before 2001, the academy could not document that the FBI had identified all laboratories housing samples of the Ames strain. In addition, the academy found, the bureau did not provide the subpoenaed laboratories sufficiently precise instructions on how to prepare the samples or on the number of spores or cells required, and it did not supervise the preparation of the samples in these laboratories. Such methods suggested to the academy that the laboratories might not have used procedures consistent enough with one another to allow comparison, nor was there sufficient certainty that the subpoenaed laboratories produced samples of all the Ames strains in their possession.

The academy report also notes that it is not clear how the FBI made scientific decisions to pursue or not pursue some investigative routes or scientific approaches. For example, Lawrence Livermore National Laboratory’s analysis of the letter material revealed the presence of tin and iron that might have provided useful chemical signatures pointing to certain production methods. Yet, the academy found no evidence that this possibility was discussed or pursued by the FBI. Nor were alternative explanations for some findings apparently explored. For example, the FBI concluded the mutations found in the anthrax letters and Ivins’s RMR-1029 flask, pointed to a common origin. But the mutations could have been the result of parallel evolution rather than derivation, the national academy’s report says, and this possibility was not thoroughly investigated. The academy also raised concerns that material from some of the letters was not analyzed to determine whether it included the four genetic markers used in comparing the samples.

Finally, the academy report notes that new technologies, such as “next generation” DNA sequencing, could have been more fully exploited to produce a more precise profile of the anthrax in the letters, strengthening the link with the RMR-1029 sample — or, perhaps, disproving it.

Recommendations for future investigations. Much of the academy’s report emphasizes the importance to biological investigations of a rigorous scientific approach — specifically a clearly defined decision-making process and well-defined methods. Methodology is particularly important; future attacks may involve agents that are not as well characterized as the Ames strain of anthrax, and the method used in the attack might not allow the recovery of a sufficient amount of agent for analysis. The academy report makes several recommendations to correct the organizational and methodological problems seen in the Amerithrax investigation.

As the academy report notes, “a scientific study is more than a series of well-executed experiments. The planning and decision-making used during the study are essential components of the science and can determine its outcome.” The report therefore recommends that in future investigations, an oversight organization should be created at the outset to “advise on procedures, strategies, and protocols, [and] help with new methods of scientific approaches.” This organization should be composed of experts not directly involved in the investigation and independent from the investigating organization, thereby fostering objective oversight and ongoing review of data results.

To ensure the thoroughness of future scientific investigations, the academy also suggests that relevant government agencies create and periodically update protocols and experimental designs for collecting and analyzing samples, preserving evidence, and otherwise buttressing the strength of biological cases in court. Equally important is the creation of a repository of all evidentiary material, so it can be compared with material in future events. The report emphasizes the importance of keeping abreast of new technological developments, as well; technological advances could not only support ongoing investigations, they could help in the review of past probes. For example, new genetic-sequencing technologies that emerged in the past decade could be used today to further investigate the Amerithrax samples, even though the criminal case is closed. Similar to the DNA analysis used so effectively in murder cases, these technologies could reinforce the case against Ivins, whose 2008 suicide precluded a public review of the case. Or the new technologies could point in a new direction altogether.

Finally, it is important to ask the question of what relative weight should be assigned to scientific information, as compared to other evidence gathered during a criminal investigation. The FBI indicated that questions relating to the production of the anthrax in the letters — whether someone had the skills required or the time needed for spore preparation — were not the only criteria used to eliminate suspects. But what weight was given to these criteria? Considering that the FBI never precisely established the required skills and spore preparation time, one may wonder whether the case would have been able to withstand the scrutiny of a trial.

To be sure, the field of microbial forensics is at an embryonic stage. If there is any overarching lesson to be taken from the FBI’s investigation, it is that microbial forensics, supported by rigorous adherence to sound methods and organizational discipline, is the sine qua non for determining the perpetrator of future bio-attacks. Indeed, allowing the perception that the United States cannot identify the source of an attack only increases the likelihood that a terrorist — or a country — might conduct such an attack. Deterrence of bio-attacks depends critically on absorbing this important lesson.