Racial bias in AI is a big problem. This activist explains why

As concerns grow over racial bias in artificial intelligence, Black Lives Matter (BLM) co-founder Ayo Tometi urged the tech sector to act fast against perpetuating racism in systems such as facial recognition.

Artificial intelligence is transforming the world and can be applied in diverse sectors, from improving the early detection of diseases to sorting out data and solving complex problems, but there are also concerns around it.

"A lot of the algorithms, a lot of the data is racist," U.S. activist Tometi, who co-founded BLM in 2013, told Reuters on the sidelines of Lisbon's Web Summit.

"We need tech to truly understand every way it (racism) shows up in the technologies they are developing," she said.

The tech industry has faced a reckoning over the past few years over the ethics of AI technologies, with critics saying such systems could compromise privacy, target marginalised groups and normalise intrusive surveillance.

Some tech companies have acknowledged that some AI-driven facial recognition systems, which are popular among retailers and hospitals for security purposes, could be flawed.

On Wednesday, Facebook announced it was shutting down its facial recognition system citing concerns about its use. Microsoft said last year it would await federal regulation before selling facial recognition technology to police.

Police in the United States and Britain use facial recognition to identify suspects. But a study by the U.S. National Institute of Standards and Technology found the technology is not as accurate at identifying African-American and Asian faces compared to Caucasian faces.

Last year, the first known wrongful arrest based on an incorrect facial recognition occurred in the United States. The United Nations has cited the case, attributed to the fact that the tool had mostly been trained on white faces, as an example of the dangers posed by a lack of diversity in the tech sector.

"They (tech companies) have to be very careful because technology has the ability to expedite values that otherwise would come about more slowly," Tometi said. "But technology speeds everything up so the impact will be worse, faster."

Urging software developers to "pay attention to all details", she said they should hear Black people more.

"Unfortunately I feel like tech companies have a long way to go to build a bridge with the community," she said.

According to the digital advocacy group Algorithmic Justice League, one of the reasons why AI systems are not inclusive is the predominantly white male composition of developer teams.

One of the hundreds of AI-driven startups that attended the Web Summit, Europe's largest tech event, was Brazil's NeuralMind, which specialises in product development.

CEO Patricia Tavares echoed Tometi's concerns, saying that although AI brings benefits to society, there was a need for "legislation to make sure companies use it in a responsible and ethical way".

Not far from NeuralMind's stand, the CEO of health tracking platform Revolab, Kalinas Ovidijus, said his startup's market was Nordic and Baltic nations and that most of the data they have access to, provided by local hospitals and health centres, was on white people.

They were unsure if the platform would be able to meet the health needs of people of colour.

"We need solutions for the future, for future challenges, but those solutions need to be very inclusive," Tometi said. "They need to protect marginalised and vulnerable communities - that's their duty." (Reporting by Catarina Demony; Additional reporting by Miguel Pereira and Pedro Nunes; Editing by Andrei Khalip, Alison Willliams and Gareth Jones)