Prediction and Validation in Wargaming: Are they possible?

Prediction and Validation in Wargaming: Are they possible?

by Roger Mason PhD

Introduction

For the past three millennia, people have longed for the ability to predict the future. From 800 BC, the most sacred shrine in ancient Greece was the Oracle of Delphi. Alexander the Great consulted with the Oracle before attempting to conquer the known world. The Oracle claimed a direct connection to the Greek god of prophecy, Apollo.

For ancient decision-makers seeking the counsel of the Oracle met two important criteria.  The accuracy of the prediction was assured because it came directly from Apollo. Any issue regarding the accuracy of the prediction was blamed on the recipient’s interpretation.

The interest in developing wargames to predict the future remains the Holy Grail of wargame designers. In the early1990s scientists were predicting future wargames employing agent-based modeling could predict the behavior of terrorists. In 2022 MIT developed a system for integrating prediction functionality with a time series database.

The program is called “tspDB” (time series database). It works well at predicting future values and identifying missing data points. It has not been employed to predict human centric activities like war and terrorism.

 

Defining Predictions and Validation

Evaluating the predictive value of a wargame requires a definition of the concepts of prediction and validation. In an article titled The Paradox of Predictions author Zann Gill explores the nature of predictions. Gill observes “A prediction, according to the traditional scientific connotation, characterizes a phenomenon such that its behavior can be anticipated a prioiri.” Zann suggests there are four connotations of the term prediction.

• Absolute: passive
• Probabilistic: passive
• Possibilistic: active
• Uncertain: active.

The active nature of possibilistic and uncertain connotations fits the employment of wargames as a method for forecasting future events. The logician C.S. Pierce suggested there is an alternative to inductive and deductive reasoning. He coined the term abduction which is the ability to make inferential leaps in the development of a new idea. The use of abduction seems common in predictions regarding the outcomes of wargame scenarios.

Gill’s fourth connotation of prediction refers to uncertainty. This is the backdrop of any military operation. In scientific research absolute control is required to ensure the process and its repeatability. In military operations or wargaming control is elusive, Gill compares it to demanding a prediction of the path of innovation or invention. These processes often take unexpected turns.

Scientific validation is the process of demonstrating the predictions of scientific models supported by repeatable results. If we heat water to a certain temperature, it boils. If we drop a cannonball off a tower it will fall at a specific rate. When a repeatable process produces identical results, validation is established.

 

Wargame Predictions and Validations.

There are three reasons why games cannot predict real world outcomes or directly validate a hypothesis. The first problem is the correlation of the model. No matter how careful you are it is impossible to perfectly replicate a real-world system or situation. There are too many unknown factors to construct the perfect model.

Secondly, it’s impossible to anticipate the environmental changes that may occur during a real incident. These are events or actions that directly impact the outcome of a situation or system. There are too many variables that impact real-world conditions to simulate these changes.

Thirdly, including human decision-makers further clouds understanding of the outcome. The introduction of biases, individual problem-solving methodologies, and varying skill levels will influence the outcome of the final decision. The three factors of modeling challenges, environmental change, and human decision-making render repeatability complex and scientific validation impossible.

 

The History of Wargames and Predictions

Predicting the future with wargames has resulted in mixed results. Some games failed to account for critical aspects of the problem they were modeling. Others accurately predicted events but were ignored by their sponsors. In 1940 the Germans wargamed the logistics for an invasion of Russia. The game predicted a gradual breakdown of the German army’s ability to keep the offensive supplied as the ground forces moved deeper into Russia. The accuracy of the prediction had little impact on Hitler who ignored them.

The Imperial Japanese Navy wargamed Operation MI, the attack on Midway. The game revealed the possibility of heavy losses during the main attack. The game balance was adjusted to improve Japanese survivability. This assured a Japanese win in the game but failed to predict the outcome of the actual battle. In December of 1942 the Japanese wargamed their evaluation of Guadalcanal and very accurately predicted how the operation would unfold, and the Allied forces would react.

In the 1960s the US Department of Defense was assessing possible outcomes of the war in Southeast Asia. Their SIGMA wargames predicted a drawn-out conflict leading to a stalemate and heavy casualties. The disturbing predictions of the early games led to the game being redesigned. The changes resulted in an outcome more favorable to the US.    

 

Developing Models

If using wargames to predict the future is impossible, what is possible? Wargames can provide a glimpse of the future by narrowing the possible end states and outcomes. It is possible to offer a form of qualitative validation through face and content validation. The first step is developing an accurate model.

There is a tendency for some wargame designers to merely find an order of battle and ensure the simulated force has the appropriate weapons and abilities. Building a model involves understanding the adversaries. Equally important is the battlespace where they will operate. If your intention is to forecast future events, it is essential to develop a battlespace to support your model, understand how the adversaries in your game world think, and how their system is organized.

 

Establishing the Battlespace

Game designers want the most accurate game model they can design. This accuracy involves details including an accurate representation of the battlespace, the environmental factors within the battlespace, and time.

 

Battlespace

The battlespace represents the geographic area where the wargame will occur. Part of forecasting operational outcomes requires sufficient space for the forces to maneuver, employ weapons systems, and engage in strategies with your game.

 

Environment

The environment in the battlespace can be divided into two parts: natural and synthetic. Natural environment factors could include light, surface condition of the ground, temperature, or rain. These changes can impact operations. Synthetic environments refer to systems or conditions related to operational factors. An example might be the battlefield’s sensor environment. This environment is constantly changing as both parties deploy measures and countermeasures.

 

Time

There must be sufficient game time (player turns representing real-time) to allow  actions to mature. The representation of time can be manipulated depending on it’s impact on game play and effect on forecasting. Time can be compressed to focus events, segmented to accelerate events, and varied to manipulate action.   

 

Assessing Assumptions

The way people act and think are unique to each individual. Each person has their own bias framework, methods for solving problems, and making decisions.

People also have a set of basic assumptions which indicate how they frame reality around them. These assumptions form the basis for how a person acts and thinks. When designing a wargame about a future event it is essential to understand what critical assumptions each agent or player may possess because they form the basis for planning.

In the 2022 Russian invasion of Ukraine, Russian planners assumed that a vertical air assault on Hostomel Airport near Kyiv would provide a means to seize the Ukrainian capital. This was based on three assumptions. 1) The attack would be a surprise. 2) The Russian air assault could capture the airport before the Ukrainians could organize their defenses. 3) A ground force could reach the airport in 48 hours to reinforce the air-assault.

The results were very different than the assumptions. The Ukrainians had pre-attack intelligence. They were prepared and decimated the Russian air-assault. The secondary attack bogged down into a “40 miles of death” gridlock resulting in the destruction of the Russian relief column.

The Russians based their planning on critical assumptions but never tested the individual elements they were based on.  The game designer should take time to identify potential critical assumptions. This is possible by identifying the elements that support each assumption and by rating the probability of an element’s occurrence.

 

Profiling the Agents

The third piece in developing your model is understanding the agents who will contest the battlespace. A detailed evaluation of each agent is necessary if the game is going to offer any forecasting ability. This is beyond the analysis of many serious and commercial games where the order of battle and weapons systems capabilities are considered sufficient. This model includes the agent’s organization, what systems and networks support them, and what processes sustain them.

The next step is understanding what motivates and confounds the agents. This includes their critical assumptions. points where they are vulnerable, and uncertainty which impacts decision-making. There are six factors for each agent in your wargame.

Systems and Networks

What systems and networks are critical to the operations of each agent? In a game based on a future scenario, ground forces might have a system of autonomous weapons comprised of networks representing the parts of the system and how they are connected.

 

Critical Assumptions

Critical assumptions are things each agent or player assumes will happen. This does not mean every tactical option but things connecting factors of the scenario, the individual battlespace, and the capabilities of a particular agent. Assumptions can support beliefs that certain operations, actions, or conditions are possible (or impossible) within the game.

They become critical when the occurrence impacts ultimate success or failure, Critical assumptions are usually based on some set of facts even if incomplete. In this way they differ from merely gambling or taking a risk. It is vital to consider them in your game because they will influence the actions the players or agents may select.

 

Processes

Processes are sequential actions or steps that lead to a specific outcome. They often involve how the agent is sustained or employs a particular weapons system. Examples in a wargame might include road and rail movement or reload time on a weapons system.

 

Critical Points

Critical points are places that are sensitive and vulnerable to the agents. They may represent connection points between networks and systems. They may require resources to support or protect them. Critical points can also offer disruption opportunities for adversaries.

 

Preferences

Preferences represent the intentions and desires of the game agents. Preferences may be localized such as seizing a geographic feature or deeper like crossing a river at several points. They may represent strategic goals or a doctrinal strategy such as seizing the enemy capital with a preemptive attack.

 

Uncertainty

Uncertainty represents what the player does not know. This lack of awareness may include their environment or the size, actions, or location of an enemy. It can be static or positional. Static uncertainty may be related to a condition in the battlespace. Positional uncertainty may be resolved by seizing a location on a map.     

 

Example of a Modeling Profile

By following this process you can understand the potential adversaries in your game. Here is an historical example employing this process based on the 1942 Battle of Midway.

 

Imperial Japanese Navy Profile for the Midway Attack

 

Systems and Networks: Carrier air forces, bombardment force, and landing force.

 

Critical Assumptions: The Japanese carrier forces can neutralize the American planes on Midway and sink any American aircraft carriers. The bombardment forces can cripple the island’s ground defenses. The landing force can defeat the island defenders.

 

Processes: The carrier forces carry anti-ground and anti-ship munitions. It takes time to rearm aircraft after each strike.

 

Critical Points: Remaining undetected prior to attacking Midway Island.  Protecting the carriers from enemy air attacks is vital especially during fueling and rearmament activities.

 

Systems and Networks: Tracking the American forces employing submarines and scout planes.

 

Uncertainty: the number of American aircraft carriers and their location.

 

Preferences: Destroy the Midway air forces, sink any American carriers in the area. , and capture the island

 

This profile provides the assessment of systems and networks, critical points, critical assumptions, processes, uncertainty, and preferences.

 

Validation

Can a wargame be validated? Can the observable outcomes be replicated? Is the game  based on accepted standards of scientific observation and experimentation? The answer is no. Validating a wargame in a scientific sense is impossible other forms of evaluation may offer a type of validation.

The Association for Qualitative Research says “Validity refers to how well a scientific test or piece of research actually measures what it sets out to or how well it reflects the reality it claims to represent.” When referring to wargames I believe the latter may be possible. Two forms of validation can be adapted to provide a form of validity to a game.

 

Face Validity

A good starting point is face validity. Face validity is a superficial and subjective assessment of the validity of a model or system. If it is superficial and subjective, what value does it have? As a starting point it is the first evaluation of your game. If the initial evaluation fails to satisfy casual observers, it is unlikely they will investigate your game any further.

Face validity is how well your model or game passes a preliminary evaluation. Three questions can help assess the face validity of a game.

1. Is the game relevant to what it represents in the real world?
2. Do the game mechanics promote a realistic model?
3. Are the game’s contents adequate to represent the system or situation you are modeling?

 

Content Validity

The second form of validity is content validity.  Content validity evaluates if the content of your game includes the relevant parts of the system or situation you are modeling.

Here are three questions to evaluate your content validity.

1. What source materials is your game based on?
2. What are the critical points of the original system and how are they represented in your model?
3. How are the relevant aspects of the model supported by the game mechanics?

 

Evaluating a Game

Here is a hypothetical example. You have designed a new Guadalcanal game. Your game simulates a well-known series of battles and actions from the Second World War in the Pacific. A casual set of observers scan your game. They ask, ‘Where is Henderson Field and the allied air units?’ Why did they ask? Anyone familiar with this battle knows the allied airbase on Guadalcanal was crucial to provide air support in defense of the island.  

Secondly, they ask what were your sources for the game research. You reply you relied on a YouTube video.  So, what is the assessment of your face and content value? On the face your model is missing a key component of the battle. For the content you used a very limited research source.

 

Playtesting

Playtesting is often like game design. Designers play through the game to ensure the mechanics work. Commercial game designers introduce some sense of balance so both players feel they have the chance to win. If you intend the game to provide some predictive value a greater attention to detail is required. There are three areas which should be evaluated to ensure the best analytic results.

 

Checking the Mechanics

The game should be playtested for mechanics. The designer should set up the game to employ each of the game’s systems to test their viability. Depending on the scenario, they may include combat, movement, logistics, indirect fire, air operations, sensor activities, and autonomous vehicle operations.

 

Testing Points of Action

The next issue of playtest evaluation is assessing the game’s points of action. These are places within the battlespace where units are likely to come in contact, attempt to seize strategic points, or need to protect lines of communication. The game designer should consider setting up opposing forces at various locations and playtest the game from each position. By playtesting each of these points the designer will better understand what can be expected during the game.

Initial Points of Contact

This is the location or locations where units from each side can come into initial contact. This is important because of the possibility of being drawn into larger combat.

 

Position of Strategic Value

These are locations in the battlespace that both sides wish to possess. They could be features like a crossroad, a town, or a river ford. During the game it is conceivable that these points will be contested.

 

Lines of Communications

These are lines connecting ground forces to logistics and communications throughout the front. The critical materials and information these lines provide guarantees they will be a target of interest.

 

 

Revelation and Emergence

Each preceding steps provides a comprehensive approach for building a wargame model capable of providing some forecasting possibility. Once the model has been developed and the game has been thoroughly playtested two new sources of data may become available.. .

 

Revelation

Revelation can be described as facts that were always present but were not understood or perceived. Most of the time these facts arise during playtesting. The validity of the revelations can be established by successive playtests of the same conditions. Revelations provide a clearer understanding of your model by carefully manipulating it.

Emergence

The concept of emergence is part of complex adaptive systems theory.  Emergence is the interactions of individual parts in a system, These interactions result in outcomes that would be impossible to predict or produce from the individual parts. An easy analogy is mud. Dirt is not mud. Water is not mud. When they are combined, you get mud. The result is something new that had not existed before.

Emergence in wargaming takes the form of possibility. Playtested interactions result in a set of outcomes. These outcomes may vary in detail but they often result in general results. In this case emergence can provide a form of forecasting. An example might be a series of playtests where Player A is always aggressive resulting in heavy losses. What might emerge is the fact aggressive action will result in casualties offering opportunities for a counterattack.

Emergent behavior is difficult to predict and will only occur after playtesting various scenarios in the game. This is where repeatability may be possible. A wargame will not provide the scientific exactness of an experiment but emergent properties or conditions can be confirms by repeated playtesting.

Although wargames cannot predict the future or be scientifically validated, we can begin developing some level of forecasting possibilities by learning from revelations and observing emergent behavior. Zann Gill, suggests, “Possibility emerges as recognized figure from the confused ground of unrealized possibility, possibility revealed by discovery, and invention. Emergence describes what occurs; revelation describes our perception of what occurs.

 

Summary: What is possible?

We began by asking two questions: can wargames provide predictive insights about real world operations and can these predictions be validated. The answer remains that wargaming any situation cannot assure absolute results in forecasting future events. We also observed that wargames cannot be scientifically validated since the number of variables makes the outcomes unrepeatable. The value of wargames is narrowing the possibilities for forecasting future events. Face and content validity helps to verify game content.

One possibility may come from the development of predictive modeling. Predictive modeling is the process of creating a model using past and present information and events. Instead of definitive predictions predictive modeling provides a form of forecasting that helps to narrow the number of potential outcomes such as emerging risks or environmental change. Combining these techniques with wargaming methods may improve the predictive outcomes of wargames.  

Reaching a point where even preliminary forecasting is possible demands a level of assessment that is not required in many professional or commercial wargames.

It is vital to understand each part of the opposing forces. From their foundational assumptions their preferences and actions are based on, to the systems, processes, and critical points that define their operational being. These details ultimately demonstrate content validity in your design.    

Playtesting is critical. Verifying your game mechanics are sound is just the start. Comprehensive playtesting requires more than merely playing the game from beginning to end. It involves evaluating inner game scenarios like where the opposing forces are likely to meet, the strategic points they will fight for, and how they will protect their  logistics and lines of communications.

Once all of these things have been tested the final step is repeated play of the entire game. With a detailed understanding of the background, an evaluation of critical actions within the game, and repeated testing a general form of the future begins to emerge in possibilities with an increasing factor of probability.

Your wargame may not provide detailed answers but a probable set of outcomes that narrows the set of possibilities. Just like a ship captain navigating a harbor in fog there will never be perfect information but possibly enough to prevent an disaster. Futurist Karl Schroeder summed it up by saying,” Foresight is not about predicting the future. It’s about preventing surprise.”  

Prediction in wargaming remains desirable but elusive. The risks of “getting it wrong” are great. Mark Twain was probably correct when he said “Prophesy is a good line of business, but it is full of risks.