We talked a lot about how haptic technologies can improve safety in the automotive industry. Haptics can improve the Human-Machine Interface (HMI) user experience of touchscreen infotainment systems and also the driver’s awareness with various types of warnings and alerts. It can also improve safety by reducing the driver’s eyes-off-the-road time when completing a secondary task on a touchscreen.
This is all nice, but we’re heading towards a future where cars will be driving autonomously. There will be no need for haptic safety features since the car will drive by itself, right? You couldn’t be more wrong! Haptic technologies will play a predominant role in autonomous car HMI. We can even argue that they will be more useful in the car automation evolution than in basic driving applications.
First, we need to understand the different levels of driving automation and how they change the way the driver needs to communicate with the car and vice versa. As you’ll see, autonomous vehicles very often share responsibilities with the driver. It will be even more important for the driver to know what exactly what the vehicle controls so he can handle the rest.
When someone talks about autonomous car driving, we often picture a car without any controls. No steering wheels, no pedals, etc. Just a car with seats so the passenger can relax while travelling on the road. While this utopian concept may come to life on day, we’re far from it as we are speaking now. The automotive industry is investing a lot of resources into autonomous driving and a lot of advanced technologies are developed. To map the evolution of those technologies, the automotive industry created an autonomous driving scale to measure the different levels of automation and how they involve the different technologies available and the driver responsibilities:
The driver needs to perform every task in order to drive the vehicle, like accelerating, braking, steering, etc. This is normal driving as we know it.
The first level of automation means the vehicle can assist with some basic functions. The driver still performs all operational tasks in order to control the vehicle. Speed cruise control is a level one automation feature.
The second vehicle automation level offers driving assistance features and allows the driver to disengage from some tasks. The driver needs to stay completely aware of the vehicle surrounding environment and must be ready to take control of the vehicle at any moment. Level 2 automation features include autonomous driving between lanes, for example.
Starting at level 3, the vehicle uses sensors (like cameras and LiDAR systems) to control all the environment monitoring operations. The driver still needs to be fully aware of the surrounding environment. While the vehicle can operate critical functions (like braking) by itself when the conditions are safe, it may need the driver to intervene.
At levels 4 and 5, the vehicle can fully operate autonomously. It can brake, accelerate, turn, monitor the roadway, determine when to change lanes, use road signals and respond to unexpected events.
The driver can still play a role in level 4 automation. The autonomous driving system will require to start under the driver’s control and send a notification when the conditions are safe for a control transition.
The final stage of automation requires no human attention or intervention. Level 5 autonomous vehicle doesn’t need pedals, brakes or a steering wheel. The vehicle controls all critical tasks and can analyze and monitor its surrounding environment and can identify unique events and driving conditions.
As you can see, depending on the vehicle automation level, the driving responsibilities can switch between the driver and the car. The level of communication is very important between the vehicle and the driver. The car driver needs to know exactly when it’s possible to give control to the car and when he needs to take over the car. Communication is primordial, since it impacts everyone’s security on the road.
It’s also important to have clear communication in order to improve the driver’s trust toward the autonomous vehicle and reduce his stress levels. Let’s be honest, giving complete control of driving to an automated vehicle implies having no control on how the car handles itself on the road. While autonomous vehicles aim at reducing accident, they can’t completely eliminate the risks, and we’re letting our lives into the hands of computers. It’s completely normal to reach higher stress levels when onboarding an autonomous car. Fortunately, better communication between the vehicle and the passenger have a positive impact in order to reduce stress levels and improve trust.
This is where haptic will play its predominant role in autonomous vehicle HMI. Haptic feedback uses the touch sense, a highly efficient human sense for communication that is currently underused in modern automotive HMI. Let’s see how haptic feedback can improve communication between a human driver and an autonomous vehicle.
The most important part of autonomous vehicle HMI is to establish an efficient communication channel with the driver for safety warnings and alerts. As you saw earlier, with the different levels of automation (levels 3 and 4), the driver and the autonomous car need to always be on the same page, especially when it comes to understanding who is currently operating the vehicle.
The process where an autonomous vehicle asks the driver to take control of the vehicle is called a Take Over Request (TOR). TOR occurs when the car can’t fulfill the driving operations. Therefore, TOR is very important in autonomous driving. They trigger the control transition between the vehicle and the driver. When the vehicle requests its driver to take over, he needs to react quickly, but first, he needs to notice the TOR signal or alert. Developing better-performing HMI for autonomous vehicles is important to make sure the driver is aware that he needs to take over the vehicle driving operations.
For example, take a level 4 autonomous car driving on the highway. The car suddenly needs the driver to take over the driving responsibilities because it starts to snow, and the vehicle sensors are less efficient. The car needs to share the information to the driver in order to initiate a control transition. The driver needs to notice it and react quickly to take control of the vehicle. This is one of many examples where the driving duties can switch from the car to the driver.
The main challenge with TOR signals and alerts is the driver’s attention. Drivers have the tendency to take their attention off the road when they don’t operate the vehicle. Simply sending a visual signal, like flashing a light in the vehicle’s dashboard, won’t get noticed by the driver. There is no way the driver can react to TOR if he’s not aware of it.
This is where haptic technologies have a lot of value!
Human Factors Evaluation of Level 2 and Level 3 Automated Driving Concepts, a study conducted by the U.S. Department of Transportation (National Highway Traffic Safety Administration) compared different types of signals to alert drivers of a Take Over Request. Their studies show that using haptic feedback with visual alerts generated reaction times with an average of 1.3 seconds vs. 4.8 seconds while using only visual alerts. This means haptic feedback can reduce the TOR reaction times by almost 3.7 times!
Haptics offer better performance because the driver’s attention isn’t on the road while the vehicle is driving autonomously. The driver can entirely miss visual TOR alerts. Fortunately, our touch sense is always “on”, meaning we will always feel and notice haptic feedback. Plus, humans react quickly to touch signals. Therefore, the autonomous car needs to incorporate tactile HMI to avoid catastrophic situations where a driver doesn’t answer take over requests fast enough.
Improving autonomous car HMI with haptic feedback isn’t only important for safety purposes, it’s also useful and effective for increasing the driver’s trust, to minimize error, reduce the driver’s stress level and stimulate the driver’s attention.
Level 1, 2 and 3 offer different levels of automation, from operating basic features like speed control (cruise control) to driving autonomously in specific circumstances. Haptic feedback can help minimize errors in assisted driving situations. Creating Appropriate Trust for Autonomous Vehicles, the Master thesis of Fredrick Ekman and Mikael Johansson at Chalmers University of Technology, tested different HMI to communicate between the car and the driver in assisted driving situations (level 2). Here’s the feedback they got from their testers:
During the assisted driving mode, which should not be confused with fully autonomous mode, the participants felt that it was hard to understand if the system was on or off (the same issue as in the assisted system activation event), what the system will do, and what the driver needs to do and have control over. For the question on what would give more trust for the system, several participants said that they would have wanted more information about when the system is fully activated, what it does, and in general more information about the system. They also said that they would have wanted to learn the system before first usage in order to familiarize themselves with the functions.
When asked questions regarding how they would like to have the information presented, several participants said that they would like the visual and audio information to be more clear – as one participant noticed when a beep sound occurred, “the beep was unclear, maybe use some other kind of information instead”. Participants also wanted more haptic feedback i.e. vibrations regarding upcoming events and warnings etc.
- Fredrick Ekman and Mikael Johansson, Chalmers University of Technology
In autonomous driving levels 3 and 4, the driver needs a clear and effective trigger to understand when the car is fully capable of taking over the driving operations. Haptic feedback is perfect for this use case since it doesn’t require the driver’s visual and auditory attention to send the information. Clear haptic feedback can improve the driver’s trust to let go the steering wheel and let the autonomous vehicle handle the work. It also minimizes errors where the driver could believe the car is ready to take over the driving operations while it isn’t the case. Haptic feedback is also helpful to reduce the overall driver’s stress burden. With clear tactile feedback, the driver wouldn’t suffer from a constant fear of missing a visual take over request from the vehicle.
Like we saw earlier in Take Over Request signals, haptic feedback is effective to stimulate the driver’s attention span. We previously mentioned that it’s easy for the driver to deviate its attention span from the road when he’s not operating the vehicle. Haptic feedback is an effective way to bring back the driver’s attention back to the road.
Studies show that tactile feedback has a meaningful impact on autonomous vehicle HMI, but there are many different haptic technologies to choose from and they’re not equally performing. You should look at piezoelectric actuators to produce high-quality haptic feedback with the space and power constraints in a vehicle cockpit.
We covered how piezoelectric haptic technologies perform compared to Eccentric Rotating Mass (ERM) motors and Linear Resonant Actuators (LRA). Piezo actuators can create more detailed tactile feedback, are smaller (therefore easier to integrate in the cockpit) and, thanks to our CapDriveTM Technology, consume less power. You should look at our blog on haptic technologies to learn more about why piezo actuators outperform other haptic technologies.