Is Public Wi-Fi Always Irresponsible? Sometimes It Is the Only Way to Get Online
Crisis maps have a distinctive role. They are not intended merely to show geographical space. They must help people understand a situation, make decisions and take action. They often have to do so when time is limited, information is overwhelming and mistakes may have real consequences.
Clarity in a laboratory does not guarantee usability in a crisis
This is why a map interface cannot be evaluated solely by asking whether users like it or consider it clear. Such evaluations are useful during the early stages of design, but they provide only part of the evidence required for crisis situations.
A map that appears understandable during a calm laboratory test may not perform equally well when a user needs to:
- identify a risk area quickly,
- understand how the situation is developing,
- distinguish essential information from secondary details,
- choose an appropriate course of action under time pressure.
What ecological validity means
Our research therefore focused on the ecological validity of crisis-map evaluation. In other words, we examined whether the testing conditions resembled the conditions in which these maps are actually intended to be used.
We reviewed dozens of empirical studies concerning map interfaces, geovisualisation and crisis decision-making. For each study, we considered three simple but important questions:
- Was the map tested by representative users?
- Did the test include at least some conditions relevant to a crisis situation?
- Did the study measure objective performance rather than subjective impressions alone?
What current studies usually demonstrate
The result is important for practice. Many studies provide a sound basis for formative usability evaluation, which helps identify problems in the interface, information hierarchy or visual design.
Fewer studies, however, support stronger claims that a map will perform effectively in real crisis use. This creates a gap between what a map demonstrates during a calm test and what is expected from it in a demanding situation.
Three levels of crisis-map evaluation
We therefore proposed a simple framework that distinguishes three levels of evaluation.
- Standard usability testing
This focuses on interface clarity, navigation, readability and basic design problems. It is essential, but it does not by itself verify readiness for crisis use.
- Higher-fidelity testing
This introduces time pressure, more complex tasks, larger amounts of information or users with relevant professional roles.
- Field or operational validation
This examines not only whether the interface is understandable, but also whether the map can support decision-making in a realistic organisational and situational context.
Evaluation must correspond to the intended claim
This leads to a practical conclusion for cartography, geographic information systems and crisis management. The method used to evaluate a map should correspond to the strength of the claim being made about it.
A different type of evidence is sufficient when claiming that an interface is understandable to an ordinary user than when claiming that it supports emergency responders making decisions under time pressure.
- a claim about clarity requires evidence that the interface is understandable,
- a claim about effectiveness requires measurements of completion time and task success,
- a claim about operational readiness requires a realistic context, representative users and objective performance measures.
Usability is a foundation, not final proof
The purpose is not to question conventional usability testing. On the contrary, sound usability testing is an essential foundation.
For crisis maps, however, we need to distinguish more clearly between refining an interface and verifying its readiness for demanding use.
Practical conclusion
Test the map under the conditions in which it will actually be used
The quality of a crisis map cannot be assessed solely through impressions of its interface. The stronger the claim made about its usability, the more closely the evaluation must resemble the real environment, users and tasks for which the map is intended.
