NVIDIA's Autonomous Cars Learn By Watching You Drive

Real World Testing

NVIDIA, the company known for its advanced graphics processing units and deep-learning systems, is making its debut in a very strange field indeed—the realm of driverless tech. Their unusual ideas for developing self-driving technology might just be what’s needed to usher the dream of autonomous vehicles into reality.

Their approach? Design a system that watches human drivers and learns all that’s needed from them—a step that eschews the laborious and difficult process of deploying legions of computer programmers to write billions of lines of code designed to anticipate the chaotic conditions of actual driving.

The features of NVIDIA’s system are available for perusal in the document “End to End Learning for Self-Driving Cars” at arxiv.org. Essentially, their idea was to outfit a training car with three cameras and two of the company’s DRIVE PX autonomous car computers, and then let the whole system observe human drivers for three days.

The enormous quantities of 3D data acquired by the camera systems are assimilated and stored in the two onboard GPUs. After the road tests and training sessions were completed, this information was fed into NVIDIA’s Torch 7, a machine learning system that breaks down the huge mass of data into digestible chunks and intelligent rules to guide autonomous systems.

Driving in the Cloud

Armed with this knowledge, the driverless tech attained 98-100% autonomy. In real world models, a car will have just one camera and computer, and possess all the knowledge it needs to drive itself. When the camera detects an unexpected situation, for which it hasn’t the necessary training, it alerts the driver to take over.

Then, the system enters “learning mode”—it watches the driver’s response and learns from it. That information is uploaded to the cloud, and enrolled in subsequent software updates so all vehicles using NVIDIA’s system can learn from the experience.

Of course, there are doubts—if something crops up that the system hasn’t handled before, it alerts the driver and transfers control. But when driving, these situations often occur in a split second. Can the situation be detected and the driver be alerted and take over in time?

Either way, this tech is a step in the right direction, and another chain in that link between today’s “dumb” cars and the effortlessly self-driving cars of tomorrow.