Teleoperation for Autonomous Vehicles

Autonomous Vehicle technology is maturing, and many driverless fleet operators are moving towards scaling and commercialization. Yet, there are still (and always will be) edge cases the AV does not know how to handle. These may include edge cases such as a conflict between traffic laws and the road situation, unclear road markings, interaction with law enforcement, unidentified objects, and more.

Teleoperation (remote operations, “human in the loop”, Tele-assist, and other names)  enables a remote human to be called upon to supervise and when needed, intervene, allowing AVs to accomplish their mission.

There are many forms of teleoperation, that fall into two major categories:

Remote Driving - remote operator is in control

The remote operator is in charge of the Dynamic Driving Task (DDT), or more specifically the combined activities of perception, localization, and defying target steering, and target speed. Even under remote operation, some actions may be performed by the vehicle.

Tele-assist - a helping hand, when needed

Autonomous vehicles have made huge advances and can easily handle most situations in their Operational Design Domain (ODD). In cases when the autonomous driver does not know how (or is not allowed) to handle, it asks a remote operator for assistance. The remote operator provides guidance, additional information, or even a policy override, based on which the autonomous driver can proceed to complete its mission.

Within these two high-level categories, there are several different teleoperation modes. The difference is a progression of the division of tasks between the remote operator and the Autonomous Driver. 

Driverless Vehicle Teleoperation Taxonomy

Modes of Teleoperation​

Another way to look at the Teleoperation taxonomy is according to the level of human involvement in the task of operation.

The choice of which mode of teleoperation to use depends on factors such as:

  • The driving environment
  • Vehicle form factor
  • Typical speed
  • Vehicle occupancy (people or goods)
  • Vehicle autonomous capabilities

Teleoperation and safety

In all but the T0, responsibility for safety, i.e. safety of people in and around the vehicle and of equipment is placed on the autonomous driver. This requires a certain level of perception and autonomy capable of maintaining safety distance, performing minimal risk maneuvers, and emergency braking

Teleoperation modes in different use cases

Teleoperation modes can change during the drive, as required for the specific case and circumstances. For example, an autonomous vehicle may require remote driving with low-level control (T1) when exiting a parking lot, then Tele-Assist high-level control (real-time) when driving on a specific road while most of the ride will require Tele-Assist path planning or supervision. A specific edge case encountered may again require one of the “lower” modes of teleoperations to extract the vehicle from the scenario.

Scroll to Top