Education and Skills

Alibaba made a smart screen to help blind people shop and it costs next to nothing

Lenovo smartphones are displayed during a news conference announcing the company's annual results in Hong Kong May 21, 2014. Chinese personal computer (PC) and smartphone maker Lenovo Group Ltd aims to sell 100 million smartphones and tablets in the coming year, the company said during a conference call with media on Wednesday. REUTERS/Bobby Yip (CHINA - Tags: BUSINESS SCIENCE TECHNOLOGY TELECOMS) - GM1EA5L1CT701

Smart Touch isn’t just for fingers. Image: REUTERS/Bobby Yip

Rita Liao

Just a few years ago, Li Mengqi could not have imagined shopping on her own. Someone needed to always keep her company to say aloud what was in front of her, who’s been blind since birth.

When smartphones with text-to-speech machines for the visually impaired arrived, she immediately bought an iPhone. “Though it was expensive,” Li, a 23-year-old who grew up in a rural village in eastern China’s Zhejiang province, told me. Cheaper smartphone options in China often don’t have good accessibility features.

Screen readers opened a plethora of new opportunity for those with visual impairments. “I felt liberated, no longer having to rely on others,” said Li, who can now shop online, WeChat her friends, and go out alone by following her iPhone compass.

Reading out everything on the screen is helpful, but it can also be overwhelming. Digital readers don’t decipher human thoughts, so when Li gets on apps with busy interfaces, such as an ecommerce platform, she’s bombarded with descriptions before she gets to the thing she wants.

Over the past two years, Alibaba’s $15 billion R&D initiative Damo Academy has been working to improve smartphone experience for the blind. Its latest answer, a joint effort with China’s prestigious Tsinghua University, is a cheap silicone sheet that goes on top of smartphone screens.

Li is among the first one hundred visually impaired or blind users to trial the technology. Nothing stands out about the plastic film – which cost RMB 0.25 or 3.6 American cents each to produce – until one has a closer look. There are three mini buttons on each side. They are sensory-enabled, which means pressing on them triggers certain commands, usually those that are frequently used like “go back” and “confirm”.

“It’s much easier to shop with the sheet on,” said Li. Having button shortcuts removes the risk of misclicking and the need for complex interactions with screens. Powering Smart Touch is human-machine interaction, the same technology that makes voice control devices possible.

Alibaba’s $1 “Smart Touch” plastic sheet helps smoothen smartphone experience for the visually impaired.
Image: Alibaba

“We thought, human-machine interaction can’t just be for sighted people,” Chen Zhao, research director at Damo Academy told TechCrunch. “Besides voice, touch is also very important to the blind, so we decided to develop a touch feature.”

Smart Touch isn’t just for fingers. It also works when users hold their phones up to the ears. This lets them listen to text quickly in public without having to blast it out through speakers or headphones. Early trials of ear touch show a 50 percent reduction in time needed to complete tasks like taking calls and online shopping, Alibaba claims.

Emotions also matter. People with visual disabilities tend to be more cautious as they fumble through screens, so Smart Touch takes that into account. For instance, users need to double-click on the silicone button before a command goes through.

Have you read?

At the moment, Smart Touch works only on special editions of Alibaba’s two flagship apps, e-commerce marketplace Taobao and payment affiliate Alipay . The buttons automatically take on different functions when users switch between apps.

But Zhao said she wanted to make the technology widely available. Some tinkering with existing apps will make Smart Touch compatible. The smart film requires more testing before it officially rolls out early 2019, so Damo and Tsinghua have been recruiting volunteers like Li for feedback.

“Unlike with regular apps, it’s hard to beta test Smart Touch because the blind population is relatively small,” observed the researcher, but embedding the technology in popular apps could speed up the iteration process.

There’s also the issue with distributing the physical sheets. According to state census, China had around 13 million visually impaired people in 2012. That’s about one in a hundred people. However, they are rarely seen in public, as a post on China’s equivalent of Quora points out.

One oft-cited obstacle is that most roads in China aren’t disability-friendly, even in major cities. (In my city Shenzhen, blind lanes are common but they often get cut off abruptly to make way for a crossing or a bus stop.)

Damo doesn’t plan to monetize the initiative, according to Zhao. She envisions a future where her team could give out the haptic films — which can be mass produced at low costs — for free through Alibaba’s expanding network of brick-and-mortar stores.

Time will tell whether the accessibility scheme is more than public relations fluff. Initiatives around corporate social responsibility have mushroomed in China in recent years. They have come under fire, however, for being transient because many merely pander to the government’s demand (link in Chinese) for corporate ethics overlook long-term impact.

“The technology is ready. It just takes time to test it on different smartphones and bring to users at scale,” said Zhao.

Don't miss any update on this topic

Create a free account and access your personalized content collection with our latest publications and analyses.

Sign up for free

License and Republishing

World Economic Forum articles may be republished in accordance with the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Public License, and in accordance with our Terms of Use.

The views expressed in this article are those of the author alone and not the World Economic Forum.

Stay up to date:

The Digital Economy

Share:
The Big Picture
Explore and monitor how The Digital Economy is affecting economies, industries and global issues
World Economic Forum logo

Forum Stories newsletter

Bringing you weekly curated insights and analysis on the global issues that matter.

Subscribe today

13 leaders on the books that changed how they work, live and lead

David Elliott

December 19, 2024

From classroom to career: Building a future-ready global workforce

About us

Engage with us

  • Sign in
  • Partner with us
  • Become a member
  • Sign up for our press releases
  • Subscribe to our newsletters
  • Contact us

Quick links

Language editions

Privacy Policy & Terms of Service

Sitemap

© 2024 World Economic Forum