Tesla’s Autopilot, which the automaker is trying to turn into a self-driving system, is going to detect potholes and make mini-maps to remember them, according to a new comment from CEO Elon Musk.
In order to achieve full self-driving, a system would have to be able to handle a wide range of different scenarios, including different weather and road conditions.
These conditions, like potholes, can sometimes be difficult for human drivers to handle, and some people find it improbable that self-driving systems will be able to appropriately navigate them.
Tesla is leveraging its large customer fleet equipped with Autopilot hardware to capture data on those corner cases and teach its neural network to handle them.
As we recently reported, Tesla is going through “a significant foundational rewrite in the Tesla Autopilot.” As part of the rewrite, Musk says that the “neural net is absorbing more and more of the problem.”
It will also include a more in-depth labelling system.
He added in an interview on the Third Row Podcast:
3D labeling is the next big thing — where the car can go through a scene with 8 cameras and paint a path in real-time, then label the path in 3D. This is probably a two to three orders of magnitude improvement in labeling efficiency and accuracy.
When asked if Tesla could use this new 3D labelling to create maps of potholes, Musk answered in the affirmative.
It means that Tesla vehicles could build maps that detect potholes and help other Tesla vehicles avoid them.
Musk didn’t say when the update is coming, but he did say that the Autopilot rewrite is almost completed.
This is a big deal.
To be fair, plenty of human drivers are not attentive or quick enough to avoid some potholes, but a full self-driving system will be held to much higher standards and will have to avoid potholes
Last year, I logged in over 300 km (186 miles) of driving on the latest Autopilot update available in my Model 3 over a few hours.
The system performed extremely well, autonomously steering on the highway and adapting the speed to the traffic around the electric car.
With one hand on the steering wheel applying light pressure, Autopilot controlled the vehicle from point A to B almost without any input from me.
The only times I had to take over was to avoid large potholes, which are unfortunately a plague in Quebec, especially during the spring:
If Tesla can figure this out, it will be a significant step forward. And of course, the same system can be replicated for other things than potholes.