8 million books and 65 million articles later, this AI has found the secret to happiness

By analyzing 8 million books and 65 million newspaper articles, researchers have created the first-ever “index of national happiness” going back to 1820.

Key findings include from the index include:

Money helps: Increases in national income do generate increases in national happiness but it takes a huge rise to have a noticeable effect at the national level.

Longer lives matter: An increase in longevity of 1 year had the same effect on happiness as a 4.3% increase in GDP.

War is costly: One less year of war had the equivalent effect on happiness of a 30% rise in GDP.

There are low moments: In post-war UK, the worst period for national happiness occurred around the appropriately named “Winter of Discontent.” In post-war USA the lowest point of the index coincides with the Vietnam War and the evacuation of Saigon.

“What’s remarkable is that national subjective well-being is incredibly resilient to wars,” says coauthor Thomas Hills, professor at the University of Warwick and the Alan Turing Institute.

“Even temporary economic booms and busts have little long-term effect. We can see the American Civil War in our data, the revolutions of ’48 across Europe, the roaring 20’s, and the Great Depression. But people quickly returned to their previous levels of subjective well-being after these events were over.

“Our national happiness is like an adjustable spanner [wrench] that we open and close to calibrate our experiences against our recent past, with little lasting memory for the triumphs and tragedies of our age.”

“Aspirations seem to matter a lot: after the end of rationing in the 1950s national happiness was very high as were expectations for the future, but unfortunately things did not pan out as people might have hoped and national happiness fell for many years until the low-point of the Winter of Discontent,” says coauthor Daniel Sgroi, professor at the University of Warwick.

Past national happiness and policy today

Governments the world over are making increasing use of “national happiness” data from surveys to help them consider the impact of policy on national wellbeing.

Unfortunately, data for most countries are only available from 2011 onward, and for a select few, from the mid 1970s. This makes it hard to establish long-term trends, or to say anything about the main historical causes of happiness.

In order to tackle this problem, the researchers took a key insight from psychology—that more often than not what people say or write reveals much about their underlying happiness level—and developed a method to apply it to online texts from millions of books and newspapers published over the past 200 years.

Their findings appear in Nature Human Behavior.

Words with stable meanings

The method uses psychological valence norms—values of happiness that can be derived from text—for thousands of words in different languages to compute the relative proportion of positive and negative language for four different nations (the USA, UK, Germany, and Italy). The research team also controlled for the evolution of language, to take into account the fact that some words change their meaning over time.

“For example, the word “gay” had a completely different meaning in the 1800s than it does today,” explains coauthor Chanuki Seresinhe of the Alan Turing Institute. “We processed terabytes of word co-occurrence data from Google Books to understand how the meaning of words has changed over time, and we validate our findings using only words with the most stable historical meanings.”

They validated the new index against existing survey-based measures. One theory as to why books and newspaper articles are such a good source of data is that editors prefer to publish pieces that match the mood of their readers.

“Our index is an important first step in understanding people’s satisfaction in the past. Looking at the Italian data, it is interesting to note a slow but constant decline in the years of fascism and a dramatic decline in the years after the last crisis,” says coauthor Eugenio Proto of the University of Glasgow.