Can AI Infer Personality from Text? Only If We Are Very Careful

The idea that AI can infer a user’s personality from text is attractive. It could help adapt communication, recommendations or the style of support. At the same time, this is an area in which it is easy to make exaggerated claims.

A good classifier score is not enough

In our research, we therefore approach estimates of personality style with caution. We do not ask only whether a classifier achieves a good score. We also ask:

• whether the reported score is genuinely reliable,
• whether it was inflated by information leakage in the data,
• whether the model expresses its uncertainty appropriately,
• whether its output is sufficiently useful for the intended form of system adaptation.

Accuracy alone is therefore not sufficient evidence that a model has captured something psychologically or practically meaningful.

When the model learns clues rather than personality

Public datasets used for personality-type prediction often contain data leakage. Instead of learning personality style, a model may learn words, abbreviations, community-specific phrases or direct references to the personality types themselves.

The result may look impressive in a table of metrics without showing that the model has captured anything genuinely useful. The data must therefore be cleaned, obvious type-related expressions removed, and the decline in performance measured once the easiest clues have been eliminated.

Uncertainty matters as much as the correct answer

A second issue is calibration. A model may not only be wrong; it may also be excessively confident in an incorrect answer. This is particularly important in adaptive systems.

An inaccurate recommendation in an optional example is one thing. Changing the form of important expert advice on the basis of an uncertain personality estimate is another. In some situations, it may be safer not to adapt at all or to ask the user directly which communication style they prefer.

A bounded adaptation hypothesis

We therefore work with the idea of a bounded adaptation hypothesis. A style inferred from text should not be treated as a psychometric truth about a person. It can only be a cautious signal that the system uses when:

• it has passed appropriate validation checks,
• the level of confidence is sufficient,
• the adaptation is low-risk,
• the user can change or reject the adaptation.

This approach is important for knowledge-based expert systems, recommender systems and AI-based support more generally. Personalisation can be useful, but it should never be applied blindly.

It is not enough to say that a model predicts something. It must be shown that the prediction is sufficiently robust, appropriately calibrated and genuinely valuable in the intended use compared with simpler alternatives.

Perhaps it is better simply to ask

Realistic alternatives are often overlooked. There may be no need to infer personality from text when the user can simply be asked whether they prefer:

• a concise answer,
• a detailed explanation,
• step-by-step guidance.

In many systems, an explicit choice may be safer, more transparent and more reliable than hidden inference.