So what’s next? Well, now that AI can develop its own “limbic system”, how can it integrate into an agent’s behavior (or “mind”)?

Here’s one way we could try:

But were these synthetic emotions genuine?
We validated LOVE with an emotional attribution study. In it, humans could consistently identify the agent’s emotions, closely matching what LOVE had self-learned. 📊🤖
We also compared them to five academic accounts. 📚✨

Analytical comparison revealed matches with 7 basic emotions, one appearing in two intensity levels.
And yes, during landing episodes, these emotions showed situationally coherent patterns: elicitation, decay, natural transitions, and even moments of confusion.

We needed “interpretability mappings” to put names to those patterns. Here’s the LOVE 2:5x6 mapping we used, as “ideal references”.

Can you match the learned patterns to the 30 ideal references? 👇

So, would it actually learn “emotions” from the rewards and state-values observed by an agent?
When tested on a classic RL case study (LunarLander-v2), it learned these 8 patterns.

But what did they mean???

Here’s an overview of the framework, including an emotional encoder (observations -> emotion vector) and an interpreter (emotion vector -> word).
We used an autoencoder because they can learn great low-dimensional representations of complex patterns.

Alternative continuation: a competitor arrives first! After the initial surge, expectations drop... but not all the way down: the agent isn’t giving up on that sweet prize just yet!

Yes, the agent would probably feel “anger” now...

Now an opportunity arises, bringing a wave of “yummy” positive rewards. Expectations soar first, then gradually settle as rewards materialize.

Would the agent feel... “excitement”, followed by “satisfaction”?

Think of this struggling agent: low energy, scarce opportunities, every step costs a little (tiny negative rewards). The sequence for rewards (green) and expectations (orange, the value function) would look like Pattern 1.

Would the agent feel... “concern”?
(feel ≈ synthesize)

Ok, first things first: do emotions matter for AI? Well, if our best reference for an AGI (human 🧠) is any clue, the answer is yes.

It turns out that animal minds—including our own—are fundamentally driven by emotions, as painstakingly explored by Darwin in... 1872!

I'd love to hear your thoughts. Anyway, my personal guess is we’re still far from hearing from an agent:

“Now I know I've got a heart 'cause it's breaking...”
(Tin Man, The wizard of Oz)

There are many theories to pick from... Plutchik’s wheel of 8 primary emotions x 3 intensities + 8 mixes is very popular and self-explanatory. I love its continuous (non-categorical) nature.

Paul Ekman’s popular BET (Basic Emotion Theory, 1971) describes 6 basic cross-cultural ones: happiness, sadness, disgust, anger, fear, surprise. But in 1999 he extended the list to 15!

I'd love to hear your thoughts. Anyway, my personal guess is we’re still far from hearing from an agent:

“Now I know I've got a heart 'cause it's breaking...”
(Tin Man, The wizard of Oz)

There are many theories to pick from... Plutchik’s wheel of 8 primary emotions x 3 intensities + 8 mixes is very popular and self-explanatory. I love its continuous (non-categorical) nature.

Btw, animals can display great intelligence too and since Darwin (The Expression of the Emotions in Man and Animals - 1872) they also officially have emotions (who on Earth could have doubted it!?)

Ok, so now, how do you “give a heart” to the machine?

Furthermore, when expressed externally, emotions also provide a primordial language that allows inter- and intraspecies communication. (Isn’t it fantastic how watching a nature documentary we can tell how animals feel from sounds and body language?)

I’ve worked on the topic of emotions + AI for a while now with the goal to “give the Tin Man” a heart.

I haven’t... yet :)

But I did learn a lot of “emotional engineering” along the way that you may find interesting or inspire you new RL approaches.