Can a Machine Care? Wrong Question for Your Business
"Can a machine care?" is the wrong question for a business to be asking. No credible researcher in the field claims a machine feels anything. The MIT lab that founded affective computing describes its work as helping systems recognize and respond to emotion, never as giving them feelings of their own. The right question is narrower and far more useful: does the system reliably notice when a customer needs human help, and does it actually get someone there?
Why is "can it care" the wrong question?
It is unanswerable and irrelevant to the outcome you actually want to produce. Rosalind Picard, who founded affective computing at the MIT Media Lab in 1995 and received the 2026 IEEE Medal for her wearable-sensing work in the field, has spent three decades building systems that detect physiological and behavioral signals correlated with emotion, not systems that experience emotion (IEEE Corporate Awards, 2026). The distinction is not semantic. A thermostat "detects" temperature without feeling hot. A sentiment model detects text patterns correlated with frustration without feeling frustrated. Asking whether an AI cares is waiting for a category error to resolve itself.
What a business can actually govern is behavior. You control what the system does when it detects a signal that a human situation is unfolding. That is answerable, testable, and worth designing around.
What does the science say about reading emotion at all?
The science urges caution. A landmark 2019 review in Psychological Science in the Public Interest, co-authored by five researchers across psychology and computer vision, examined the accumulated evidence on facial expressions. The authors concluded that how people communicate anger, disgust, fear, happiness, sadness and surprise "varies substantially across cultures, situations, and even across people within a single situation" (Barrett, Adolphs, Marsella, Martinez & Pollak, Psychological Science in the Public Interest, 2019). There is no reliable signature mapping a given expression to a given internal state (Barrett et al., 2019). If human experts working from rich, in-person cues cannot reliably decode emotion from expression alone, a system working from text or a voice channel demands deep skepticism.
This is the uncomfortable truth vendors tend to skip. The more confidently a product claims to "understand" your customer's emotional state, the more it is overselling. That is scientific fact, not skepticism for its own sake.
What should a business actually hold a vendor accountable for?
Hold them accountable for escalation discipline, not empathy claims. The useful question is direct: when sentiment signals frustration, risk, or uncertainty, does the system hand the conversation to a person, and how fast? That is an operating duty, not a customer courtesy. Context, emotion, risk, and uncertainty each independently justify pulling in a human. A system that only escalates on the loudest, most obvious cases is under-designed. We apply this same logic one level deeper in the rule against confirming unverified actions, and explore its boundaries in the limits of sentiment scores.
| Question you might ask | Why it's the wrong frame |
|---|---|
| "Does the AI understand how my customer feels?" | Unanswerable, and not what changes outcomes |
| "Is the AI empathetic?" | Empathy is a human capacity, not a measurable AI property |
| "When sentiment or risk crosses a threshold, does a human get looped in, and how fast?" | Testable, auditable, and the thing that actually protects the customer |
What does regulation say about this line?
Regulators draw this line firmly. The EU AI Act's Article 5 prohibits AI emotion-inference systems in workplaces and schools except for narrow safety and medical exceptions, an explicit acknowledgment that treating inferred emotion as settled fact carries real risk to people (EU AI Act, Article 5). The Montréal Declaration for a Responsible Development of AI states the same principle more broadly: systems that touch human wellbeing carry a duty of caution scaled to potential harm, not to how impressive the technology sounds (Montréal Declaration for Responsible AI, 2018). Neither document asks whether a machine cares. Both ask what happens next when a signal appears.
Judge your vendors on that standard. Ignore how warmly they describe their AI. See how AiPRL frames human escalation as an operating duty rather than a feature, and read how a sentiment signal should function as sentiment as early warning.