How 3D sensors are changing our lives

This post is part of a blog series with Young Scientists ahead of the Annual Meeting of the New Champions 2015, which takes place in Dalian, China, from 9-11 September. Michael Bronstein works in the Department of Computer Science, Technion, Israel Institute of Technology and is a World Economic Forum Young Scientist. In this blog he talks about his research and taking 3D sensing beyond its early application in gaming into a whole range of practical uses. 

3D sensors were popularized by the Xbox Kinect. Aside from gaming, give us three other areas of life that it is transforming?

Three-dimensional sensing is a technology that could bring new capabilities in a lot of fields. In security, it means fast and reliable recognition of a face or palm geometry. In the automotive industry, it means built-in mechanisms that monitor a driver’s behaviour, alerting them if their attention wanders and deciding (in the case of an accident) whether to deploy the airbags.

In virtual and augmented reality applications, 3D sensing provides an element of “self-presence”, allowing users to see themselves in the virtual world. The inability to see their own body was one of the complaints from the people testing out Oculus virtual-reality glasses.

I could go on: in the academic community there are a lot of exciting ideas related to 3D that need to be commercialized. I expect that in the next few years we will start seeing some revolutionary products.

What are the practical hurdles preventing this technology becoming more mainstream? Power usage and clunky hardware have been mentioned as limiting factors in the past.

I think it is (or, at least, is in the process of) becoming mainstream. The video-gaming application for which Kinect was designed was a relatively clunky device that consumed a lot of power. Nevertheless, it was the first mass-produced, affordable 3D sensor. With my colleagues Alex Bronstein, Ron Kimmel and Sagi Ben Moshe, I was involved in an Israeli start-up company called Invision, where we took things one step further, developing technology that would allow us to shrink a 3D sensor to fit inside the display bezel of a laptop or tablet.

Invision was acquired by Intel in 2012. Today you can walk into an electronics store and buy a laptop that has our 3D camera in place of the standard webcam. Lenovo, Dell, Fujitsu, HP, Asus, Acer – all these brands now have 3D-enabled laptops. The number of applications and uses for these new capabilities grows every day.

What is the most exciting aspect of your work?

The field has been evolving fast over the past years. I remember how much time we spent 10 years ago, trying to convince investors that 3D was commercially interesting. Today, everyone seems to be convinced. From a scientific point of view, dealing with 3D data is very exciting, as it requires the confluence of various disciplines: differential geometry, numerical optimization, harmonic analysis, linear algebra, among others. A few years ago, 3D geometry analysis used to be a niche in the computer vision community, but now the number of publications related to 3D data has exploded.

Looking at the wider area of science and technology, which areas excite you most?

I am interested in “deep learning”, which has achieved extraordinary results in image analysis and computer vision. In the geometry-processing community, these methods are not widely used or known, mainly because of some fundamental differences between “flat” objects or images and “curved” 3D shapes. Recently we’ve been working on the extension of convolutional neural networks (one of the learning paradigms popular in computer vision) to such objects, achieving remarkable progress in notoriously hard tasks such as deformable 3D shape matching. Furthermore, many of the methods we use for the analysis of 3D geometric data can be applied to analyse more abstract structures such as web-scale graphs or social networks.

What do you hope to achieve from your participation at AMNC?

I was at the New Champions meeting in Tianjin last year. The people the Forum brings together are amazing, and I’m looking forward to days of interaction with the smartest people on the planet.

Have you read?
Why scientists need to think like entrepreneurs
How can artificial intelligence make us healthier?
Can a computer tell what you’re thinking?

Author: Michael Bronstein works in the Department of Computer Science, Technion, Israel Institute of Technology and is a World Economic Forum Young Scientist. He will be participating at the Forum’s Annual Meeting of the New Champions, which is taking place in Dalian, China, 9-11 September.

Image: 3D glasses are displayed at the Internationale Funkausstellung (IFA) consumer electronics fair at “Messe Berlin” exhibition centre in Berlin, September 3, 2010. REUTERS/Christian Charisius