This AI judges you based on the biases we've instilled it with

First impressions

When we see someone for the first time, we make internal snap judgements about them. We can’t help it, we’re just judgmental like that. After looking at the person for just a few seconds, we might note their gender, race, and age or decide whether or not we think they’re attractive, trustworthy, or kind.

After actually getting to know the person, we might find out that our initial perception of them was wrong. No big deal, right?

Well, it’s a very big deal when you consider how our assumptions could shape how the artificial intelligence (AI) of the future make increasingly important decisions.

In an effort to illustrate this issue to the public, researchers from the University of Melbourne created Biometric Mirror.

Public perception

Biometric Mirror is an AI that analyzes a person’s face and then displays 14 characteristics about them, including their age, race, and perceived level of attractiveness.

To teach the system to do this, the Melbourne researchers started by asking human volunteers to judge thousands of photos for the same characteristics. This became the dataset Biometric Mirror referenced when analyzing new faces. Because the information these volunteers provided was subjective, so was Biometric Mirror’s output. If most of the human respondents thought people with beards seemed less trustworthy, that would influence how the Biometric Mirror judged people with beards.

The ethics of AI

To use Biometric Mirror, a person just has to stand in front of the system for a few seconds. It quickly scans their face and then lists their perceived characteristics on a screen. The AI then asks the person to think about how they’d feel if it shared that information with others. How would they feel if they didn’t get a job because the AI ranked them as having a low level of trustworthiness? Or if law enforcement officials decided to target them because they ranked highly for aggression?

“Our study aims to provoke challenging questions about the boundaries of AI. It shows users how easy it is to implement AI that discriminates in unethical or problematic ways which could have societal consequences,” lead researcher Niels Wouters said in a press release. “By encouraging debate on privacy and mass-surveillance, we hope to contribute to a better understanding of the ethics behind AI.”

Eliminating bias

A system as biased as Biometric Mirror could have major consequences as AI becomes more widely used and makes increasingly important decisions. And this isn’t just some future possibility, either; we’re already seeing examples show up in today’s systems. While researchers work on figuring out ways to ensure future systems don’t contain those same flaws, it’s important that the public consider the potential impact of biased AI on society. Biometric Mirror could help them do just that.