Teleoperation for Autonomous Vehicles
Autonomous Vehicle technology is maturing, and many driverless fleet operators are moving towards scaling and commercialization. Yet, there are still (and always will be) edge cases that the AV does not know how to handle. These may include conflicts between traffic laws and the road situation, unclear road markings, interaction with law enforcement, unidentified objects, and more.
Teleoperation (AKA, remote operations, “human in the loop,” Tele-assist, etc.) enables a remote human to be called upon to supervise and—when needed— intervene, allowing AVs to accomplish their mission.
The many modes of teleoperation:
Remote Driving: the remote operator is in control
The remote operator is in charge of the Dynamic Driving Task (DDT), or more specifically, the combined activities of perception, localization, defying target steering, and target speed. Even under remote operation, some actions may be performed by the vehicle.
Tele-assist: a helping hand, when needed
Autonomous vehicles have made huge advances and can easily handle most situations in their Operational Design Domain (ODD). In cases that the autonomous driver does not know how (or is not allowed) to handle, it asks a remote operator for assistance. The remote operator provides guidance, additional information, or even a policy override, based on which the autonomous driver can proceed to complete its mission.
Within the two high-level categories, there are several different teleoperation modes. The difference is a progression of the division of tasks between the remote operator and the Autonomous Driver.
Driverless Vehicle Teleoperation Taxonomy
Route planning and control
Another way of looking at the Teleoperation Taxonomy is to rank it according to the level of human involvement in the task of operation.
Teleoperation and safety
In all but the T0, responsibility for safety—of people in and around the vehicle and of equipment—is placed on the autonomous driver. This requires a certain level of perception and autonomy capable of maintaining safety distance, performing minimal risk maneuvers, and emergency braking.
Teleoperation modes in different use cases
The choice of which mode of teleoperation to use depends on factors such as:
- Driving environment
- Vehicle form factor
- Typical speed
- Vehicle occupancy (people or goods)
- Vehicle autonomous capabilities
Teleoperation modes can change during the drive, as required per the specific case and circumstances. For example, an autonomous vehicle may require remote driving with low-level control (T1) when exiting a parking lot, then Tele-Assist high-level control (real-time) when driving on a specific road, while most of the ride will require Tele-Assist path planning or supervision. A specific edge case encountered may again require one of the “lower” modes of teleoperations to extract the vehicle from the scenario.