That strange line
Recently, I treated myself to a nice rental car including all the extra features. It was going to be a long ride to Northern Germany, so I deserved it as I told myself. This car included adaptive cruise control (ACC), automated lane keeping (ALK), traffic jam assist (TJA), lane change assist (LCA) and more of these catchy three letter abbreviations. ‘It can do basically everything on the highway’, the rental guy said. Really? He couldn’t explain the difference between ‘basically everything’ and ‘everything’. Hmm.
I told my wife, with this car I could simply release the steering wheel on the highway, at least for a short time. ‘Don’t you dare’, she said. Luckily, this was a solo trip, so no supervision here. After waving my wife goodbye, I gradually turned on as many functions as I could find. I had to stop once to consult the manual and find out that some of the functions were on anyway – nothing to turn on. Others were only on at certain speeds. Now I am the oddball that likes to read a manual, but what will John Average make of this?
It all worked fine. My dashboard was illuminated with icons like a Christmas tree. Starting with the obvious ACC to relax my legs, I turned on TJA and LCA to release the wheel. It seemed to work. It did follow the lanes, it was aware of a slow truck and helped me to change lanes. It did slow down when some dodo tried to hit me when cutting in. Yes! I started to relax and focus on the radio – sorry, in-car entertainment. It had found my phone somehow and now it wanted to play the music from my phone. Too many options – where is the radio?
Figure 1 A50 rush hour lane (source: wegenwiki.nl)
It was by then that I had reached the A50 between Arnhem and Apeldoorn. This is where they have one of these rush hour extra lanes – essentially the emergency lane which turns into an extra lane at rush hour. After 10 kilometres in that extra lane, the first exit came into view. I was just about to change radio stations when a sudden jolt to the right frightened the hell out of me. Heartbeat 170 and up. Suddenly I was on the exit I did not want. After taking over the wheel and parking, I got out of the car to get my heartbeat under control. What had just happened? I called my friend who had recommended the car to me. Oh, he said dryly, I know that spot. The emergency lane has no lining at the exits, so the car just follows the continuous line. You simply have to hold the wheel at that spot. Thanks, pal, how am I supposed to know?
Is this safe?
Naturally, I should have paid attention and kept my hands on the wheel. However, everything from the rental guy to the advertising and the car interior suggests: relax, we’ll handle it for you. It is like a permanent lullaby on a long ride. But how can I be sure this car is safe when I drive away with it?
This is where the EU HEADSTART project comes in. It develops a consistent way to test and collect safety evidence for a vehicle (see previous blog by Nicolas Wagener). This means collecting situations (or ‘scenarios’) and defining what is relevant for this car. It means limiting the automated functions to a well-defined and well-tested type of road situation (the ‘operational domain’). It also means finding the right combination of various types of tests and then testing until you are very sure. And agreeing which extremely rare cases (‘edge cases’) can be ignored.
Two topics are interesting for today: what is relevant for the safety of a given automated car? And: how about communicating cars?
What is relevant?
The intuitive answer would be: anything the car could possibly meet on the street in its life. This does not fly with today’s assisted driving functions since they are simply not designed to handle all this. Also: ‘in its life’ means a new situation might occur tomorrow that was not anticipated in the design. The concept of the ‘Operational design domain’ (ODD) is very useful here (SAE J3016): it describes the operating conditions under which the car is designed to work. This can be road type, weather, etc. Essentially you put a fence around the subset of real-world driving that your vehicle can handle and define everything outside irrelevant. To paraphrase: everything within the ODD is not odd but relevant.
So now the new challenge is to handle everything the car meets in the ODD. By driving around and collecting scenarios we can smoothly collect common scenarios. But knowing this car might be bought by millions of people who each drive it at least 50 000 km, surely our users will find more scenarios than our data collection team can. Also, it is not ethical to ask our test drivers to look for critical situations on the road. So we are still left with a snag: how to systematically collect rare scenarios? And how rare should a situation be to become irrelevant? Should we design a car that can handle a bicyclist that ends up on the highway? And an airplane that lands on the highway?
Since serious accidents occur around once every 2.5 million km of driving for Germany (Wachenfeld, 2016), we need to drive a lot more to prove that safety is improved. This takes lot of test driving or a fresh approach. Using the sensors of high end production vehicles is the most powerful and consistent approach but can only be done with the previous generation of CAD functionality that is already type approved… Current approaches focus on combining and synthesizing risk triggers into edge cases – on the edge of what is possible and relevant.
Once you have all the scenarios collected, you are ready, right? Well, you need to keep collecting as their parameters change over time – with the introduction of more CAD vehicles the balance between conventional and automated driving changes. The numbers of scenarios are so high that you will need to test many of them in simulation for safety and cost effectiveness. And for complete safety assessment or type approval you need more. Sensor tests on component level are crucial and can be done in the lab. Emergency braking is most convincingly and safely tested on a test track. Human factors can be tested in a simulator, but you really want to validate all this by test driving on the public road. So, we end up with a well-balanced mix, including public road testing. Real-world traffic does not nicely stick to one isolated scenario at a time: a critical situation is usually a soup of many ingredients spiced with all the noisy distractions of the road.
Talking cars
Cars start to become more talkative: transmitting traffic jam information to navigation systems and receiving warnings from infrastructure on advised speed and roadblocks. Real-time communication between vehicles allows for fast cooperation and possibly even higher levels of safety. This raises the question how to test the integrity of the data and robustness of the wireless link. After all, you now rely on external information from a variety of infrastructure sources and multi-brand vehicles. Communication standards, interoperability and trust relations are crucial here.
HEADSTART looks at what can go wrong (failure modes) and includes communication events in scenarios to test this on the level of interaction between vehicles. Scenarios are important as a missed message (for example) may have a much higher impact in an urban congested scenario than in a nightly highway scenario.
If you want to test vehicle-to-vehicle (V2V) communication including two or more vehicles (like platooning), you need to release or approve each vehicle brand and model in the configuration, respecting its role (leading, following, etc.) and the compliance with the applicable V2V protocols.
Chipping away
This is the moment to give praise to a mostly invisible expertise in automotive: the test engineers, be it on the test track or with a simulator. If we want to find out the safety of a vehicle, safety engineers first analyse hazards and risks before the test engineer starts. This includes vehicle failures and also risks during driving when no technical failures occur (operational safety). The appropriate response to such hazards is then specified, modelled and designed. Though based on a massive body of expertise, this is still paperwork. If we want proof, we need a test engineer and a test, be it virtual of physical.
Figure 2 Chipping away at the parameter distributions
So: what to test? From the scenarios collected before, we can learn the statistical variety of all kind of scenario aspects and the corresponding parameter distributions. Combined with edge cases, we can sample a set of relevant tests that give sufficient insight into the behaviour of the tested vehicle to make a release decision. By patiently going from test to test or by running a batch of many simulations, the test engineer chips away parts of the parameter distributions until nothing is left uncovered. This hard and noble work is what gives us evidence of safety.
This blog post was produced by Sytze Kalisvaart.
Sytze Kalisvaart is a senior project manager at TNO Integrated Vehicle Safety. He is responsible for scenario database projects, the StreetWise methodology and pipeline. He has a background in system design, usability and safety in medical and automotive.