Why AI is tipping the scales in the development of self-driving cars

When people think of self-driving cars, the image that usually comes to mind is a fully autonomous vehicle with no human drivers involved. The reality is more complicated: Not only are there different levels of automation for vehicles — cruise control is an early form — but artificial intelligence is also working inside the car to make a ride safer for the driver and passengers. AI even powers a technology that lets a car understand what the driver wants to do in a noisy environment: by reading lips.

In Silicon Valley, there is a race to develop the best technology for autonomous vehicles. “It’s perhaps among the most exciting times to be talking about autonomous vehicles,” said Wharton professor of operations, information and decisions Kartik Hosanagar on a panel at the recent AI Frontiers conference in Silicon Valley. “Ten years back, most of the work with autonomous vehicles was just going on in research labs and various educational institutions.” About five years ago, only Google and a handful of companies were testing them. “Today, there’s a frenzy of activity,” he said. “Just in California, the number of companies that have licenses to do testing and operating of driverless vehicles is already somewhere in the 30 to 50 range.”

Globally, the U.S. and China are ahead in the self-driving race. Germany and Japan, despite being famous for their autos, are behind. “The key difference is AI,” said Tony Han, co-founder of China-based autonomous vehicle company JingChi. “China and the U.S. are leading in AI.” When it comes to self-driving regulations, China and the U.S. also lead. What’s driving this intense interest are three mega-trends: the rising popularity of electric vehicles, emergence of the shared economy that is powering ride-sharing firms like Uber and Lyft, and advancements in artificial intelligence. If you think about it, he said, autonomous driving is really about combining a robot driver with an electric car.

According to Han, most autonomous vehicle firms are developing technology that is suitable for what he calls a level 4 roadster. There are five levels of automation in self-driving cars. Level 1 is the most minimal, with a typical feature being cruise control that has been around for years. Level 5 is the most advanced, with the vehicle being fully autonomous. Level 4 is a notch below — a highly automated level where the car can operate in certain situations without driver intervention or attention, such as in specially fenced off areas or in traffic.

AI Inside the Car

Danny Shapiro, senior director of automotive at chipmaker Nvidia, said tech companies take the development of autonomous vehicle technology seriously because the stakes are high. “This is not a recommendation engine for Netflix,” he said at the conference. “The AI has to be spot on.” That means it requires “extreme” computing power and a lot of code, Shapiro said. In the self-driving vehicle’s trunk are powerful computers and graphics processing units doing deep learning to parse all the data coming in — to determine such things as whether the object ahead is a person, another car, a fire hydrant and so on.

Even if it will take some time for fully autonomous vehicles to hit the market, AI is already transforming the inside of a car. Front-facing cameras can identify people in the vehicle and track the driver’s eye position to see whether he is falling asleep or distracted — and even read the driver’s lips. Sensors and cameras outside the car work with interior technology to enhance safety. For example, the car warns audibly that there is “cross traffic danger” if another vehicle is about to run a red light. It can also say things like “Careful! There is a motorcycle approaching the center lane!” to alert the driver in case he or she wants to do a lane change. “There’s going to be a whole host of guardian angel-type features even if we’re not fully self-driving,” Shapiro said.

Indeed, a major goal of self-driving companies is to make driving safer. Human error is responsible for 94% of car crashes, said Jeff Schneider, senior engineering manager at Uber and a research professor at Carnegie Mellon University. He noted that half of the mistakes leading to accidents were due to recognition errors — the driver was not paying attention or did not see something coming. The other half was the result of a decision error: The driver was going too fast or misunderstood the situation.

According to Schneider, self-driving vehicles can address these two types of errors. Problems of recognition would be mitigated by using sensors, radar, cameras, Lidar (a remote sensing system) and other tools. The cars can see 3D positioning of objects and other things around them, receive 360-degree camera views in high resolution and access other pertinent data such as velocities of objects. Meanwhile, sophisticated computing systems analyze the landscape to make the right driving decisions.

One way to help accuracy is by incorporating redundancy in the systems. For example, if a road sign were somehow obscured, measures are put in place to make sure the self-driving car does not get confused. Schneider said the car’s own map would inform it that there is a road sign at that location. Also, these vehicles go through enormous amounts of data to train them to operate under various conditions such as snow, rain, sleet and floods. Autonomous vehicle companies even use computer-generated conditions to train the car to drive through such things as a blinding sunset. “Using a rack of servers, we can generate over 300,000 miles [of driving] in just five hours, and test algorithms on every paved road in the U.S. in just two days,” Nvidia’s Shapiro noted.

To be sure, these are complicated tasks for the car. “Put yourself in the [position] of the person writing code” who has to account for people crossing the street, other cars on the road, billboards, traffic signs ahead and lanes for cars, bikes and pedestrians, among others, Schneider said. “You have absolute chaos.”

Safety and Security

To skeptics who see a fully autonomous vehicle as a pipe dream, it would be helpful to look back at how far autonomous vehicles have come, Schneider said. As early as the 1980s, Carnegie Mellon University’s NavLab project already was equipping vans with computers and sensors for automated and assisted driving. “It was the age of robotics when the rule was to keep the video running just in case something good happens,” he said. In 1995, the university’s “No Hands Across America” drive from Pittsburgh to Southern California was 98% autonomous and included a 70-mile stretch without human intervention, Schneider said.

In 2000, the university moved to off-road vehicles. The new things added to the roadsters were GPS and Lidars to make it easier to pinpoint objects and get around them. Seven years later, at the DARPA Grand Challenge, a contest for autonomous vehicles, a major development was the addition of good maps that provided a full reconstruction of the environment. “AI took a step forward,” Schneider said. CMU won the contest. It was also at this point that Google recognized the potential of autonomous vehicles and started its self-driving project, he said. Since then, AI, machine learning and deep learning have gotten even better.

Still, will consumers feel comfortable riding in a self-driving car? Based on Uber’s experience testing autonomous vehicles in Pittsburgh and Phoenix, Schneider said, the public seems to be open to riding in them. While there was some concern initially that people would be scared of these cars, “what we found was exactly the opposite,” he said. For example, since riders cannot choose a self-driving Uber, some customers would chase these vehicles while calling for rides in hopes of landing the car.

However, what could put a damper on the development of mass market self-driving cars is the business model. For now, it’s still more economical to own a car than take Uber everywhere. “If you just run the numbers, financially it’s not cheaper to do that than to own your own car,” Schneider said. “Once autonomous vehicles work and they’re everywhere … it won’t make sense to own a car.”