This interactive map tracks your vocal emotions

The spontaneous sounds we make to express everything from elation (woohoo!) to embarrassment (oops) say a surprising amount about what we’re feeling, research shows.

To prove that a sigh is not just a sigh, scientists conducted a statistical analysis of listener responses to more than 2,000 nonverbal exclamations known as “vocal bursts” and found they convey at least 24 kinds of emotion. Previous studies of vocal bursts set the number of recognizable emotions closer to 13.

The results, which appear in the American Psychologist, demonstrate the findings in vivid sound and color on the first-ever interactive audio map of nonverbal vocal communication.

Click here visit the online audio map, then move the cursor across map to hear exclamations linked to 24 emotions. Note that the audio works more slowly on mobile devices.

“This study is the most extensive demonstration of our rich emotional vocal repertoire, involving brief signals of upwards of two dozen emotions as intriguing as awe, adoration, interest, sympathy, and embarrassment,” says senior author Dacher Keltner, a psychology professor at the University of California, Berkeley, and faculty director of the Greater Good Science Center, which helped support the research.

Huh? And Gasp!

For millions of years, humans have used wordless vocalizations to communicate feelings that we can decode in a matter of seconds, as this latest study demonstrates.

“Our findings show that the voice is a much more powerful tool for expressing emotion than previously assumed,” says lead author Alan Cowen, a PhD student in psychology.

On Cowen’s audio map, you can slide a cursor across the emotional topography and hover over fear (scream), then surprise (gasp), then awe (whoa), realization (ohhh), interest (ah?), and finally confusion (huh?).

Among other applications, the map can help teach voice-controlled digital assistants and other robotic devices to better recognize human emotions based on the sounds we make, Cowen says.

As for clinical uses, the map could theoretically guide medical professionals and researchers working with people with dementia, autism, and other emotional processing disorders to zero in on specific emotion-related deficits.

“It lays out the different vocal emotions that someone with a disorder might have difficulty understanding,” Cowen says. “For example, you might want to sample the sounds to see if the patient is recognizing nuanced differences between, say, awe and confusion.”

Nonverbal Signals

Though limited to US responses, the study suggests humans are so keenly attuned to nonverbal signals—such as the bonding “coos” between parents and infants—that we can pick up on the subtle differences between surprise and alarm, or an amused laugh versus an embarrassed laugh.

For example, by placing the cursor in the embarrassment region of the map, you might find a vocalization that you recognize as a mix of amusement, embarrassment, and positive surprise.

“A tour through amusement reveals the rich vocabulary of laughter and a spin through the sounds of adoration, sympathy, ecstasy, and desire may tell you more about romantic life than you might expect,” Keltner says.

2,000 Vocal Outbursts

Researchers asked 56 male and female professional actors and non-actors from the United States, India, Kenya, and Singapore to respond to emotionally evocative scenarios and recorded more than 2,000 vocal bursts.

Next, more than 1,000 adults recruited via Amazon’s Mechanical Turk online marketplace listened to the vocal bursts and evaluated them based on the emotions and meaning they conveyed and whether the tone was positive or negative, among several other characteristics.

A statistical analysis of their responses found that the vocal bursts fit into at least two dozen distinct categories including amusement, anger, awe, confusion, contempt, contentment, desire, disappointment, disgust, distress, ecstasy, elation, embarrassment, fear, interest, pain, realization, relief, sadness, surprise (positive), surprise (negative), sympathy, and triumph.

For the second part of the study, researchers sought to present real-world contexts for the vocal bursts. To do this, they sampled YouTube video clips that evoke the 24 emotions established in the first part of the study, such as babies falling, puppies being hugged, and spellbinding magic tricks.

This time, 88 adults of all ages judged the vocal bursts extracted from YouTube videos. Again, the researchers categorized the responses into 24 shades of emotion. They then organized the full set of data into a semantic space onto an interactive map.

“These results show that emotional expressions color our social interactions with spirited declarations of our inner feelings that are difficult to fake, and that our friends, co-workers, and loved ones rely on to decipher our true commitments,” Cowen says.

Additional researchers from Washington University in St. Louis and Stockholm University contributed to the work.