A Map That Works in Calm Conditions May Fail in a Crisis
Crisis maps have a distinctive purpose. They are not intended merely to represent space. They must help people understand a situation, make decisions and act—often when time is limited, information is overwhelming and mistakes may have real consequences.
Clarity in a laboratory does not guarantee usability in a crisis
This is why a map interface cannot be evaluated solely by asking whether users like it or find it clear. Such evaluation is useful during the early stages of design, but it provides only part of the evidence needed for crisis situations.
A map that appears understandable during a calm laboratory test may not necessarily perform well when users need to:
- identify a risk area quickly,
- understand how the situation is developing,
- distinguish essential information from secondary details,
- select an appropriate course of action under time pressure.
What ecological validity means
Our research therefore focused on the ecological validity of crisis-map evaluation. In other words, we examined whether the testing conditions resembled the conditions in which these maps are actually intended to be used.
We reviewed dozens of empirical studies on map interfaces, geovisualisation and crisis decision-making. For each study, we considered three simple but important questions:
- Was the map tested by representative users?
- Did the test include at least some conditions relevant to a crisis situation?
- Did it measure objective performance rather than subjective impressions alone?
What current studies usually demonstrate
The findings are important for practice. Many studies provide a sound basis for formative usability evaluation—that is, for identifying problems in the interface, information hierarchy or visual design.
Studies less frequently support stronger claims that a map will perform effectively in real crisis use. This is where the gap emerges between what a map demonstrates in a calm test and what is expected of it in a demanding situation.
Three levels of crisis-map evaluation
We therefore proposed a simple framework that distinguishes three levels of evaluation.
- Conventional usability testing
This focuses on interface clarity, navigation, readability and basic design problems. It is essential, but does not by itself establish readiness for crisis use.
- Higher-fidelity scenario testing
This introduces time pressure, more complex tasks, larger amounts of information or users with relevant professional roles.
- Field or operational validation
This examines not only interface clarity, but also the map’s ability to support decision-making within a realistic organisational and situational context.
Evaluation must match the intended claim
This leads to a practical conclusion for cartography, geographic information systems and crisis management. The way a map is evaluated should correspond to the strength of the claim being made about it.
Evidence that an interface is understandable to an ordinary user differs from the evidence needed to claim that the map supports emergency responders making decisions under time pressure.
- a claim about clarity requires evidence that the interface is understandable,
- a claim about efficiency requires measurements of task time and success,
- a claim about operational readiness requires a realistic context, representative users and objective performance measures.
Usability is the foundation, not the final proof
This does not mean that conventional usability testing should be questioned. On the contrary, good usability testing is an essential foundation.
For crisis maps, however, we need to distinguish more clearly between refining an interface and validating its readiness for demanding real-world use.
Practical conclusion
Test the map as it is intended to be used
The quality of a crisis map cannot be judged solely from impressions of its interface. The stronger the claim about its usability, the more closely the evaluation must resemble the real environment, users and tasks for which the map is intended.
