Self-driving cars are no longer a thing of science fiction but have become science fact. You get to sit back and catch up on work or your favorite show while your car gets you from A to B. Unfortunately, the technology is still in its infancy, and much is still to be learned about this fantastic technology that’s designed to make our drives easier. Unfortunately, unlike our human brains, the artificial neural networks inside self-driving cars can’t “remember” what roads they’ve driven and the scenery they’ve taken in, which can be potentially dangerous in adverse weather conditions, even if the car has driven that road before.
In hopes of overcoming this constraint, researchers at Cornell University have assembled three concurrent papers, two of which were presented at the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2022). The studies suggest a way to help self-driving cars create “memories” of previous experiences and use them in future navigation.
"The fundamental question is, can we learn from repeated traversals?" said senior author Kilian Weinberger, professor of computer science. "For example, a car may mistake a weirdly shaped tree for a pedestrian the first time its laser scanner perceives it from a distance, but once it is close enough, the object category will become clear. So, the second time you drive past the very same tree, even in fog or snow, you would hope that the car has now learned to recognize it correctly."
To carry out their research, the team repeatedly drove a car along a 15-km loop in and around Ithaca, New York a total of 40 times over an 18-month period. The car was equipped with LiDAR (Light Detection and Ranging) sensors meant to capture the differing environments throughout its travels, which include highway, urban, and campus surroundings, along with changing weather conditions such as sun, rain, and snow, as well as the times of day. The project was led by doctoral student Carlos Diaz-Ruiz, who successfully collected over 600,000 scenes into the dataset that are then stored on a virtual map using various types of software, like how the human brain stores a “memory”.
"It deliberately exposes one of the key challenges in self-driving cars: poor weather conditions," said Diaz-Ruiz. "If the street is covered by snow, humans can rely on memories, but without memories a neural network is heavily disadvantaged."
The goal of this research is to ultimately reduce the cost to develop autonomous vehicles, as they still rely on costly human annotated data, unfortunately. The research could also make autonomous vehicles more efficient by using their “memories” to help navigate locations they might use the most.
As always, keep doing science & keep looking up!
Sources: Cornell Chronicle
