By Prof. Amnon Shashua
As we march towards a driverless future, we at Mobileye have continued to lead the industry with new innovations that will not only enable fully autonomous vehicles (AVs), but will also make human-driven cars safer than they have ever been. Over the years, I am proud that we have achieved many industry firsts: camera and radar fusion in 2007, pedestrian-detection warning in 2010, camera-only forward-collision warning in 2011, camera-only automatic cruise control (ACC) in 2013, hands-free assist in 2015, crowd-sourced HD-mapping in 2016, the Responsibility-Sensitive Safety (RSS) safety model in 2017 and, most recently, a “vision zero” horizon through a novel preventive system using RSS.
It is said that imitation is the sincerest form of flattery, and our innovations have not gone unnoticed with many embracing the same concepts that we pioneered. One industry player in particular habitually follows our lead and today I would like to set the record straight on its latest imitation.
Let us consider the recent past. After Mobileye announced the world’s first crowdsourced mapping technology – Road Experience Management™ (REM) – in 2016, Nvidia announced a solution the following year that claimed to do the same. When Mobileye coined and introduced L2+ in 2017 as a new category of driving automation that uniquely applied our REM technology to driver assistance systems, again Nvidia followed suit and announced its L2+ offering in 2019.
Our most recent innovation, RSS, was published in an academic paper in 2017. We openly shared all the technical details and mathematics behind RSS because we believe that the safety of automated vehicles should not be proprietary, and that the industry should collaborate with governments on what it means for an AV to drive safely.
The response to and support of RSS has been tremendous. Baidu and Valeo have publicly signed on. China ITS has approved a work group tasked with standardizing RSS for the China market. And we have engaged with governments and standards organizations around the world on RSS. What’s more, dozens of research papers have cited RSS, contributing to the public discourse on this important topic.
We’ve always said that we believe RSS is an excellent starting point for verifiable safety assurance of automated vehicle decision-making. We’ve openly invited the entire industry to contribute their ideas on how to improve RSS. We were pleased when Nvidia heeded this call and reached out to us in 2018 about a collaboration on AV Safety. We were puzzled when Nvidia backed out of the proposed partnership.
Imagine our surprise last week when Jensen Huang, CEO of Nvidia, announced a “first-of-its-kind” safety model for AVs. Curious to see what “first-of-its-kind” innovation Nvidia had created, we eagerly read the publicly released white paper about Safety Force Field (SFF), only to have the eerie feeling that we were looking in the mirror.
If imitation is the sincerest form of flattery, then Nvidia must think very highly of us. Based on the information that has been made available, it is clear Nvidia’s leaders have continued their pattern of imitation as their so-called “first-of-its-kind” safety concept is a close replica of the RSS model we published nearly two years ago. In our opinion, SFF is simply an inferior version of RSS dressed in green and black. To the extent there is any innovation there, it appears to be primarily of the linguistic variety.
To illustrate Nvidia’s latest attempt to emulate Mobileye’s technology leadership, let me first describe RSS using the technical terms (in Italics) that are defined in our publicly published materials.
RSS defines a safe longitudinal and a safe lateral distance around the vehicle. When those safe distances are compromised, we say that the vehicle is in a Dangerous Situation and must perform a Proper Response. The specific moment when the vehicle must perform the Proper Response is called the Danger Threshold.
SFF defines identical concepts with slightly modified terminology. Safe longitudinal distance is instead called “the SFF in One Dimension;” safe lateral distance is described as “the SFF in Higher Dimensions.” Instead of Proper Response, SFF uses “Safety Procedure.” Instead of Dangerous Situation, SFF replaces it with “Unsafe Situation.” And, just to be complete, SFF also recognizes the existence of a Danger Threshold, instead calling it a “Critical Moment.”
RSS covers occluded agents such as vehicles hidden behind buildings and pedestrians hidden behind cars. So does SFF.
RSS defines reasonable expectations on the behavior of other agents, such as the expected maximum braking. So does SFF.
RSS defines Proper Responses for both structured and unstructured roads. So does SFF.
RSS defines a “response time” of how long it will take the AV to respond to a change in situation. So does SFF, though it is renamed “reaction time.”
In fact, every one of the five “common sense” rules defined by RSS are replicated in SFF:
- Do not hit someone from behind.
- Do not cut-in recklessly.
- Right-of-way is given, not taken.
- Be careful of areas with limited visibility.
- If you can avoid an accident without causing another one, you must do it.
Before we spell out the similarities in more detail below, let’s talk about what an AV safety model is really about. At the heart of the construction lies the concept of a “Proper Response” in RSS terminology (“Safety Procedure” in SFF terminology). In our opinion, to really implement an AV safety model, one must explicitly specify this Proper Response – which Mobileye has done. We invested a great deal of thought to make RSS comply with three crucial properties:
- Sound: The Proper Response of RSS complies with the common sense of human judgement.
- Usefulness: RSS enables agile driving policy.
- Efficiently verifiable: it is possible to comply with RSS rules in a computational efficient manner.
Without an explicitly defined Proper Response, no practical implementation is possible. We attempted to evaluate SFF’s so-called “Safety Procedure,” but we were unable to find it in the SFF paper, except for one simple example that, in our opinion, is too simplistic and yields a non-useful driving policy. The SFF paper appears to be non-transparent where the most important ingredient is missing from the model. We encourage Nvidia to be more transparent about the details of its Safety Procedure so that the industry is better able to evaluate its safety and efficacy. And although we enjoy flattery as much as anyone else, we hope that SFF’s Safety Procedure is based on independent innovation as opposed to imitation of Mobileye’s technology leadership. But don’t take my word for it, let’s let the text speak for itself:
This analysis clearly shows that SFF is a close replica of RSS. It is indisputable that RSS was, in fact, the “first-of-its-kind” safety model for AV’s when we announced it in 2017, and we believe it is still the leading solution today.
At Mobileye, we believe in technology innovation, not linguistic innovation. We have openly invited and are enjoying active collaboration with industry and government partners around the globe. It is unfortunate that rather than collaborate with us, Nvidia felt it necessary to follow us yet again, creating confusion where there could have been cohesion. Mobileye has invested enormous resources to develop RSS, and Mobileye has obtained intellectual property rights to protect these investments.
Professor Amnon Shashua is senior vice president at Intel Corporation and president and chief executive officer of Mobileye, an Intel company.