Wargaming: how to turn vogue into science

In early March 2017, a US congressional committee called a hearing about Russian military capabilities in Europe. Ever since Russia’s military intervention in Ukraine in 2014, NATO countries have been on edge that a similar scenario could happen in the Baltic Republics of Estonia, Latvia, and Lithuania. The congressmen gathered in the hearing room were deeply concerned about whether NATO forces could stand up to a Russian invasion. David Shlapak, a RAND Corporation defense researcher, had a clear message for them: “NATO does not have in place an adequate conventional deterrent to Russian aggression.” Russian forces, Shlapak warned, could crush NATO defenses in the Baltic within 36-60 hours. If NATO had more resources such as three heavy brigades with tanks, infantry fighting vehicles, and self-propelled artillery, the alliance could withstand an attack and significantly alter Moscow’s calculus.

How did RAND come to these predictions and proposed solutions? The answer shares something in common with many people’s favorite childhood pastime: board games. Shlapak’s method for analysis was a series of operational-level wargames played by representatives from NATO allies as well as members of the US defense and intelligence communities.

Wargaming is back in vogue. Military planners have used wargames to think through strategies for centuries. Prussian military commanders began using wargaming techniques in the 1820s. The use of wargames peaked before and during World War II. Since then, wargaming techniques, which extend well beyond manual board games, have evolved to a broad range of applications. Now, with major world powers such as the United States, Russia, and China competing for dominance across a wide-range of military domains, policymakers have renewed interest in wargaming, viewing it as a tool for strategic, operational, and technological innovation.

In 2015, the US Defense Department published a memo about the need to reinvigorate wargaming. Two years later, the Pentagon requested $525 million over five years for the effort. Now Britain is following suit. The Ministry of Defence is establishing a new Wargaming Centre of Excellence to explore wargaming techniques. At the same time, a new pool of wargame suppliers–defense contractors, think tank experts, and university researchers—has emerged to cater to policymakers and others interested in wargaming.

Defense officials are once again intrigued by wargames because gaming can provide insight into areas where there is, fortunately, little empirical evidence—like nuclear deterrence and escalation. Despite scarce data, defense planners and government officials still need to make decisions on how to adapt strategy and posture in a new era of major power rivalry. Here, wargaming holds analytical potential.

According to the February 2015 memo, the Pentagon finds that wargames, “done right,” can:

• improve “understanding of complex, uncertain environments and the changing character of warfare,”
• assess “the operational impacts of technology trends, future challenges and military competition,”
• make “forces more successful in future conflicts” through the “development of new capabilities”, and “operational and organisational concepts,”
• “identify how to exploit new opportunities, hedge against discontinuities and craft long-term competitive strategies,” and
• help distinguish between “wise and unwise investment” decisions.

Britain’s vice chief of defence staff described wargaming as a powerful tool that “can deliver better understanding and critical thinking, foresight, genuinely informed decision-making and innovation.” This is a tall order that underscores the need to do wargaming right. While wargames hold analytical potential, wargaming needs to adapt to produce credible insights on consequential national security topics.

Wargaming to-date has been practised more as an art than a science. And professional wargamers design, conduct and analyse games in predominantly classified environments. This approach has led to the wide acceptance of wargaming as a method for training and development of operational concepts in the defense community. It has also confined the production of wargames to a small professional community of experts who have honed their skills through the wargaming master-apprentice guild system.

Although senior policymakers increasingly participate in table-top training exercises of simulated crises, many distrust wargaming masters to produce wargames that generate useful analytical insights–especially if these insights challenge or contradict existing beliefs. Indeed, a former senior official who drafted the 2015 Pentagon memo remarked that the number of people at the Pentagon who can do wargaming right is “exceedingly small” and “most wargames suffer from serious methodological shortcomings that limit their usefulness.”

The arts approach to wargaming  has been an obstacle to widespread acceptance in academic circles. Scholars have largely rejected wargaming tools as credible methods of inquiry. King’s College London professor Philip Sabin has been working for decades to overcome the reluctance of academics to embrace wargaming. He describes “the failure of wargaming proper to gain anything like the scholarly respectability of approaches such as mathematical modelling, operations research and game theory.”

But these attitudes are beginning to change. Political scientists, who are not wargaming masters, are publishing new articles that employ a scientific approach to wargame analysis in journals such as International Security and Science.

Analytical wargaming needs to be scientific. If wargaming tools are to underpin evidence-based analysis that informs national security and defense policy, wargames should adhere to scientific standards. Wargame producers should follow the requirements of good academic and good intelligence analysis. As former National Intelligence Council chair Tom Fingar writes, “the standard for performance [in intelligence analysis] can be no lower and arguably should be higher than those” in academic disciplines. That’s because the impacts of intelligence analysis can be “far more consequential.” The same goes for wargaming analysis.

Even within established disciplines, such as international relations, scholars diverge on which practices are scientific. But all recognise a need to distinguish between science and non-science, between the generation of factual knowledge and generation of advocacy.

Being scientific in the practice of wargaming is even more controversial. Wargaming methods can employ techniques from a range of disciplines, including political science, psychology, statistics and decision analysis. Most professional wargamers agree, in principle, that wargaming is both an art and a science. But many will vehemently oppose any suggestion that analytical wargaming should be more scientific.

Among some wargamers, the mere word science conjures up ideas of mathematical operations research methods—the same ones that made wargaming largely obsolete after World War II. But this opposition is based on a narrow understanding of the definition of science—the concept that a scientific method necessarily entails empirical testing of falsifiable hypotheses to predict the future. In fact, there are multiple ways of being scientific.

A scientific approach to wargaming should encompass both qualitative and quantitative methods from a range of disciplines. And it should include the full span of social science approaches.

American University professor Patrick T. Jackson describes a useful approach to a science of international relations that can be applied to analytical wargaming.

A science of wargaming should be built on three principles: It should be “systematic, public and intended to produce worldly knowledge.” These principles can differentiate credible analytical wargaming from advocacy and art.

Five theoretical challenges for analytical wargaming

The first step is to clearly identify a wargame’s purpose. There are analytical wargames that aim to innovate, and non-analytical wargames that aim to educate, advocate or entertain. If a wargame is analytical, that is, it intends to collect data to produce generalisable knowledge, it should follow scientific principles.

Analytical wargames start with research questions, which inform their design, data collection and analysis procedures. In contrast, educational wargames start with learning objectives. Although all analytical games educate, educational games cannot deliver credible conclusions about a research question if they do not follow proper research design procedures. Games that purport to be analytical but whose producers know the conclusions before analysing the data, are not scientific and should be considered tools for advocacy.

Wargame analyses should clearly connect premises to conclusions. Absent a clear and consistent logic, the consumers of wargame analysis can either agree or disagree with conclusions based on their pre-existing beliefs on the subject.

British defense officials have privately lamented that wargames used for strategy development are often run on tight schedules as part of an intensive defense estimate process. They are often interpreted quickly and without proper consideration of available data. Other practitioners have noted that many wargames lack a robust analysis plan that connects research questions to data collection and data interpretation. One leading US wargame designer remarked that the wargame analyst should be a subject-matter expert who simply immerses himself in the data.

These practices are wrongheaded and could lead to false learning on highly consequential issues such as nuclear use. As Jackson writes, “Facts … are accessible to anyone employing the proper procedures for disclosing them, and depend not on revelation or intuitive insight but on systematic demonstration and public, if technical, argumentation.”

Wargames should allow for peer-review. In practise, this means describing the methodology and presenting the data to allow other experts to judge the validity of game-derived insights and build on them in their own work. Classified analytical wargames should not be exempt from this requirement; relevant research communities with appropriate clearances should review their conclusions. Individuals and organisations should be able to learn from one another.

A leading US wargaming expert observed in an off-the-record discussion at King’s College London last year that professional wargamers do not have time to write text nor “have the willingness to become authors.” But to advance wargaming, wargame producers need to reference previous games to understand which designs were applied for what purposes. More importantly, wargamers need to leave a record that would allow others to judge whether their process and analysis was done right.

Progress is being made. In 2015, the Pentagon established a classified wargaming repository, which by 2018 included “750 entries on completed and future wargames” were “viewed more than 14,000 times by more than 800 people from 20 [Defense Department] organizations.” But it is unclear to what extent these records enable peer-review or whether they are used to improve wargaming. To-date, there is no unclassified archive dedicated to wargaming.

Scientific wargames should provide analytical insights into real-world problems. This is not immediately obvious with abstract gaming scenarios and simplified rule-sets. For example, NATO’s Locked Shields cyber game is built around the fictional adversary Berylia, a state located in the middle of the Atlantic Ocean. Is accurate geographical and political representation of an opponent necessary to understand escalation dynamics in cyberspace? Perhaps not.

But consider a different example – the World of Warcraft, a massive multiplayer online role-playing game. Analysis of conflict dynamics in the World of Warcraft could be both systematic and public, but would not be scientific, as it would not seek to reveal insights into real conflict.

Of course, simplification is necessary to make sense of a complex environment. But wargame producers need to provide defensible arguments that they are using abstract models with the intent to produce insights about the real world.

A scientific practice of wargaming will require constructive humility from wargamers, a willingness to improve tools for understanding and addressing consequential security challenges. To advance knowledge, wargame producers must engage in scientific dialogue with public and systematic presentation of wargaming methods and findings on real-world problems.

Since Shlapak’s testimony, NATO has bolstered its defenses in the Baltics. While some have been “highly skeptical of the assumptions underlying the RAND [wargames],” the think-tank’s public presentation of these findings has enabled a conversation among wargamers, scholars and policymakers about doing wargaming right. Good academic analysis and scientific practice are critical to further developing analytical wargaming; without that, the field could falter.