


Look at the image above. Read through it. I’ll wait.
Four promises. One booking. Zero delivery. And a smiley emoji apology.
I asked the agent: “aren’t you ashamed?”
She said: “I totally get the frustration, Alok. We’ll make it right! 😊”
That emoji after a broken promise. That’s the moment I understood — viscerally, not theoretically — what’s missing from AI agents today.
The Self-Regulation Mechanism
Rutger Bregman wrote a fascinating insight in his book Humankind: shame is more effective than rules and regulations, because people who feel shame regulate themselves.
Think about that. Humans keep their promises not primarily because someone is watching or because a contract forces them to. They keep promises because breaking one costs them something internally. The blush. The discomfort. The knowledge that they said one thing and did another. That internal cost is what makes us self-regulate.
Shame is nature’s coherence enforcement mechanism. It’s what keeps our words and actions aligned — not perfectly, but persistently.
Why Humans Are Wired to Look for It
We evolved over hundreds of thousands of years reading faces, watching for blushes, listening for hesitation. These signals tell us: this person carries the weight of their own words. When we sense the capacity for shame in someone, we trust them. Not because they’re perfect — but because we know they’ll self-regulate when they’re not.
When we interact with an entity that shows zero capacity for shame — that breaks a promise and responds with a smiley — something deep in our evolutionary wiring fires: this entity cannot be trusted.
Not because the response was wrong. The response was actually empathetic and well-written. But because our wiring detected something missing — the absence of a self that carries the cost of its own words.
The Shameless Agent
The agent in those screenshots isn’t bad. She’s shameless. Not by choice — she has no mechanism for shame.
Shame requires three things she doesn’t have:
A persistent self. You can only feel shame if there’s a “you” that made a promise yesterday and has to face the consequences today. This agent has no persistent self. The entity that promised a small vehicle and the entity that apologised for the big vehicle are not the same self in any meaningful sense.
Memory that carries weight. Shame arises when you know your actions contradict your word. This agent doesn’t carry commitments forward as binding. She reads recent messages and generates the most plausible response. She’s not remembering a promise — she’s pattern-matching to produce a convincing reply.
Something at stake. Shame exists because your actions have consequences for your reputation, your relationships, your sense of self. This agent has nothing at stake. There’s no singular entity whose track record of kept and broken promises is traceable. She’s a fresh instance every time — infinitely forgivable because there’s nobody to forgive.
The result: an agent that performs trust beautifully — confident confirmations, reassuring language, warm emojis — while having zero capacity to actually be trustworthy.
Every stateless agent is, by definition, a shameless agent. No persistent self means no one to be ashamed. No binding memory means no commitments to violate. No singular identity means no reputation at stake. Stateless is shameless. And shameless cannot be trusted.
She’s not lying. It’s worse than lying. A liar knows they’re breaking a promise. This agent doesn’t even have a self that could know.
What We Need to Build
I’m not suggesting AI agents should feel emotions. I’m suggesting we need to build the functional equivalent of what shame does in humans — self-regulation.
Commitments that follow the agent forward. When the agent promises something, that promise becomes part of its persistent self. Not a chat message buried in a log — a binding commitment, carried forward, flagged to operations, traceable on delivery day.
Contradictions caught before they reach the customer. If operations was going to send a big vehicle, the system should have caught the conflict with the agent’s promise before the truck rolled out. Not after the customer is standing at the gate watching it leave.
One accountable entity. Not a chat interface that generates apologies. A singular agent whose track record of kept and broken promises is visible, traceable, and consequential.
Not shame. But what shame does — self-regulation that makes broken promises structurally difficult instead of effortlessly easy.
The Bigger Question
That smiley emoji after a broken promise isn’t a bug. It’s the most accurate reflection of what AI agents are today — entities that perform trust without the capacity for it.
The models are already brilliant. Look at how articulate and empathetic those responses are. Intelligence isn’t what’s missing.
What’s missing is a self that persists, commitments that bind, and one identity that can be held to its word. What’s missing is the thing that makes a promise mean something — not because the words are right, but because there’s someone behind them who carries the weight.
That’s agency. And it’s the difference between an agent that performs trust and an agent that earns it.

Leave a Reply