Why Being Right Is Not Enough for AI

When evaluating AI-supported systems, we often ask whether the answer was correct. That is, of course, essential. Yet in support provided to users, students, clients or employees, correctness alone is not enough.

The same information can have very different effects

Consider two responses containing the same factual information. One is concise and action-oriented: complete the first step, then the second, and then the third. The other is longer, explains the broader context and reassures an uncertain user.

Both responses may be correct. Nevertheless, they may affect different people and different situations in very different ways.

What communication fit means

Our research therefore focuses on communication fit: the extent to which the form of a response matches the way in which a user needs to receive support.

• some users benefit from a structured procedure,
• others first need to understand the broader reasoning,
• some prefer maximum brevity,
• others need greater reassurance that they are proceeding correctly.

These are not merely minor stylistic differences. They can influence clarity, trust, satisfaction and willingness to use the system again.

What our experiments showed

We tested situations in which users evaluated communication-adapted responses in comparison with more general responses. The results indicate that an adapted communication style can improve:

• perceived clarity,
• the usefulness of the response,
• trust in the support provided,
• intention to use the system again.

In another part of the research, we also examined simulated interactions with experts and investigated whether alignment in communication style influenced how the support was perceived.

This is not about psychological labelling

Importantly, this approach does not involve psychologically labelling people. The aim is not to place users into fixed personality categories and then communicate with them according to those labels.

Communication adaptation therefore does not have to rely on a fixed psychological profile. It can respond to a specific need, the type of task, the degree of uncertainty or the user’s chosen form of explanation.

Correct content presented poorly may not help

This has practical implications for schools, organisations and digital services. A chatbot, expert system or AI assistant may provide factually correct information, but when that information is presented in an unsuitable way, users may not understand it, trust it or act upon it.

The quality of support is therefore not only the quality of its content. It is also the quality of its communication.

Finding an appropriate response

This is particularly important when users are dealing with a problem they do not understand or when they feel uncertain.

• An overly technical response may discourage the user or increase their uncertainty.
• An overly general response may fail to provide sufficiently concrete assistance.
• An excessively long response may place an unnecessary burden on the user.
• An excessively brief response may appear incomplete or untrustworthy.

Well-designed AI support must be able to manage this tension and select a form of response that matches both the situation and the needs of the person receiving it.

How future AI systems will be evaluated

Future systems will therefore not be assessed solely by whether they know the correct answer. They will also be assessed by whether they can communicate it in a way that is:

• understandable,
• usable,
• appropriate to the particular situation,
• adapted to the person who currently needs it.