Autonomous driving car (ADC) is required to have adequate safety and reliability in the complex and uncertain environment. This mini review investigates a variety of reasons for the unreliability of ADCs, describes the cutting-edge reliability modeling and analysis methods, and explores the potential solutions to improve ADC reliability. Additionally, future research concerning the reliability ADCs is proposed from mechanical, electrical, software and human aspects as well.

Keywords: reliability, robotics, autonomous driving car

As the state-of-the-art and high-risk robotics application, Autonomous Driving Car (ADC) is required operate in inclement weather, unpredictable and dynamic road conditions. Therefore, it is a crucial and challenging task to ensure its safe and reliable operation under complex circumstances. Recently, the cutting-edge navigation, localization, object recognition, state estimation and control technologies have propelled the reliability improvement of self-driving cars.^1–3 Typical reliability of ADC systems in detecting road lanes has exceeded 95%, or even near 100%.⁴ Google announced that all its future ADCs can identify almost all stop signs, constantly look out for pedestrians, traffic and other various obstacles, and will be fully driver-free with no steering wheel.⁵ Blanco et al.⁶ investigates the crash rate of ADC compared to the police-reported rate estimated from the Second Strategic Highway Research Program (SHRP 2) Naturalistic Driving Study (NDS), and the result is shown in the below Table 1.⁶ Level 1 to 3 represents the three different levels of crash in the severity decreasing order, and Level 1 represents the most severe crash.⁶ However, even if ADCs are reported to have a lower crash rate especially for more severe crashes,⁶ whether computers can operate a vehicle more reliably than human drivers is still in doubt. On Mar. 18, 2018, Uber self-driving car killed a woman pedestrian in Arizona because it failed to realize that she is a human and decided not to decelerate or swerve to avoid her.⁷ On May. 7, 2016, a Tesla ADCs failed to apply the brakes when a tractor-trailer turned left in front of that Tesla car in Florida, which caused immediate death of the driver.⁸ In the last few years, many autonomous car models have been developed, tested or even commercially manufactured.^9,10 In addition, the increasing attention has been paid to the many aspects of the ADC research.⁹ However, the research of the reliability of ADC is still not mature, and lags compared to its test and production.^9,10

Generally, the reliability of autonomous vehicles is affected by multi-disciplinary and interactive factors including hardware fault tolerance, software architectures, resilient machine learning, interaction between humans and vehicles, perception in highly dynamic and unstructured environments, testing and validation of systems operation, regulatory approaches, and etc.¹¹, which is shown in the below Figure 1.

Figure 1 Multiple Inter-disciplinary Factors to Affect ADC Reliability.¹¹

The sources of the unreliability of ADCs are summarized as below:

1. The uncertainties of manufacturing tolerance of the mechanical parts inside the robot vehicles, which enables the real ADCs to have different theoretical models, control errors and limits.¹²
2. Inaccurate measurements, incomplete information or imprecise perception of the environment.¹³ ADC requires its exteroceptive sensing systems to reliably detect obstacle and gather information. However, significant sensor noise can exist.¹⁴ Moreover, visual or perception systems can fail in low illumination environments, poor visibility conditions or heavy traffics.⁴ Consequently, only limited landmarks, such as road boundaries or lane markings can be detected.¹⁵ Thermal imagers can fail to detect stationary objects.⁴
3. Cutting-edge computer vision systems cannot achieve appropriately low error rates for autonomous navigation.³ Machine Learning has occasional reliability issues in classification or recognition, especially when the input object is beyond the training data range.³
4. In complicated conditions, such as accident areas, construction zones and unexpected scenarios, ADCs cannot reach human-level accuracies and reliability in the higher-level perception, detection, segmentation, planning, and decision-making.^3,14,16
5. Simulation and test procedures do not include all road conditions, or reflect its reliability in real driving. Except a few test vehicles, most ADCs are not exposed to road traffics, so it is difficult to identify the critical reliability gap in the actual paradigm shifts.¹⁰

Reliability is the probability that a component / system will perform its defined function under the specified conditions for the specific time period without failure.¹⁷ Time to Failure (TTF), Interval Analysis (IA),¹² Fault Tree Analysis (FTA), Root Cause Analysis (RCA), Failure Mode and Effect Analysis (FMEA) and Failure Mode Effects and Criticality Analysis (FMECA) are common methods to analyze failure, manage uncertainties and predict reliability of ADCs. Department of Motor Vehicles (DMV) defines Autonomous Miles Driven per Disengagement as an indicator to measure failure and reliability of ADCs,¹⁸ which is shown in the below Figure 2. Furthermore, Kalra et al.,¹⁹ analyzed the advanced testing, statistical and adaptive regulation solutions to demonstrate the effect of speed on ADC reliability in terms of injuries and fatalities. Berger et al.,²⁰ provided the prospect of applying software engineering to achieve safe and reliable driving on urban environments with an anticipatory ADC. Hojjati-Emami et al.,²¹ predicted the reliability of ADCs which have Advanced Driver Assistance Systems (ADAS) as well as Passive Safety Systems (PSS). Teoh et al.,²² proposed a Kalman filter and a reliability point system, where the reliability point system offers a fast and effective solution to monitor the quality of tracking the vehicles. Driankov et al.,¹³ applies fuzzy logic to analyze the solutions and reliability for the navigation of ADCs. Shi et al.,²³ utilizes generalized stochastic Petri nets to model the reliability of Redundant Vehicle Management Computer System (RVMCS) and efficient fault monitoring device(FMD) designs. Broggi et al.,²⁴ presented the use of cost maps to allow the fusion of information from several sensors and to evaluate the trajectories reliability in extreme road environment for dynamic planning. Nothdurft et al.,²⁵ explores generating the context model around ADCs, and information sharing with other traffic participants / road infrastructures as two key factors to influence the ADC reliability. Kianfar et al.,²⁶ proposed the simulation methodology and versatile architecture to verify the ADC reliability and performance in various situations. Wei et al.,¹⁵ presented the ADC research platform developed at Carnegie Mellon University to meet general requirements of ADC while addressing reliability, safety and appearance imperatives, where the sensor installation is shown in the below Figure 3.¹⁵

Figure 2 Autonomous Miles Driven per Disengagement from Quarter 4, 2014 to Quarter 4, 2015.¹⁸

Figure 3 The Sensor Installation of ADC.¹⁵

Reliability features of ADC include fault-tolerant computing systems, intuitive and smooth autonomous-manual switch, advanced navigation, environment perception, object detection and fusion, and learning strategies, fully disengagement and shutdown of the drive-by-wire and computing system on E-stop.^15,27 Moreover, redundant sensors with overlapping view areas is another reliability feature, which enable ADCs to continue running until the driver has overtaken control or a safe and complete stop is reached.^15,20 Berger et al.,²⁰ presents the combination of sensor-gathered and wirelessly received information to retrieve information from occluded and non-visible scenarios for more reliable driving decision making. Cardarelli et al.,¹⁴ introduced a multi-sensor and data fusion cloud service system to improve the reliability of the advanced navigation strategy. Milella et al.,²⁸ presented the multi-baseline stereo frames for accurate reconstruction and scene segmentation, and to increase the reliability of the perception system. Adouane et al.,¹ proposed the design and development of appropriate Multi-controller Architectures (MCAs) and mechanisms to manage these controllers' interactions, and enhance the metrics/ criteria concerning the overall control reliability in complex and dynamic environments. Fernandes et al.,²⁹ presented the simultaneous Dedicated Short Range Communications (DSRC) and infrared (IR) to improve Inter vehicle Communication (IVC) reliability of ADCs. Hardy et al.,^30,31 predict the possible future trajectories of pedestrians and bicyclists, and also use hierarchical trajectory clustering to improve the reliability of ADCs. Additionally, designing a safe shutdown mission and capability can relax the safety requirements and indirectly increase the reliability of ADCs accordingly.¹¹

Future research focuses on the appropriate integration of comprehensive mechanical, electrical, software and human driver components.⁴ Therefore, end-to-end design and deployment process can incorporate the reliability concerns of a myriad of technical specialties into a unified approach to handle any road condition.¹¹

1. The improved hardware design, software architecture, and system integration is needed to accumulate and analyze information, and generate efficient shortcuts from virtual and real sensors, to actuators in emergencies.²⁰ Sensors have many different forms, qualities, and degradation characteristics.²⁰ Software architecture should have self-awareness of the quality of the sensors, and enable graceful degradation of software system functions based on sudden sensor failures.²⁰ Therefore, research of reliable detection of system degradation will be important to reduce the possibility of vehicles unintentionally operating unsafely due to high false-negative rate, or being stranded on road due to high false-positive or false alarm.¹¹
2. When performing reliability calculations and analysis, fully fault-free redundancy is typically assumed at the start of each mission.¹¹ Any undetected fault, or even 1-2% self-diagnosis gaps undermines the benefits of redundancy and reduces the achievable reliability dramatically.¹¹ The reason is that undiagnosed failures can accumulate for the full operation time of the vehicle, so the probability of experiencing independent and un diagnosable failures during the life of a ADC will be quite high.¹¹Consequently, creating chips that can be self-tested prior to each driving cycle is critical to improve the ADC reliability.¹¹
3. Reliable and robust environment perception and learning strategy is crucial requirement to detect and understand the surrounding environment, recognize natural or artificial objects, drivable paths, and make informed and real-time decisions.²⁸ The sensor configuration, perception algorithms, localization, 360º road object detection and classification, robust path and trajectory planning can be challenging and the important future research in the ADC reliability.¹⁵ Validating inductive learning in novel environmental inputs and achieving high dependability for full fleet deployment is also a future reliability research posing significant challenges.¹¹
4. The driver’s capacities of ADCs can deteriorate by constantly changing road surroundings and conditions, which results in the potentially increasing accident rate. Like the nuclear or petroleum industry,³² human error in ADCs is hard to be predicted in an adequately reliable manner.¹⁴ Human reliability analysis in ADCs will have great potentials for obstacle avoidance and reliability improvement. Deployment of ADAS, analysis of human-machine interaction, human error, reliability and responsibility in degrading situations (i.e., the absent driver as a result of cognitive distraction) are the critical future ADC reliability research.^4,20

Autonomous Driving Car (ADC) is the cutting-edge robotics application with crucial reliability requirements to solve problems in complex situations, reason in a timely manner and react safely in unpredictable circumstances. This paper reviews comprehensive deciding factors to reduce ADC reliability, up-to-date reliability modeling and analysis methods, as well as applicable solutions to improve ADC reliability. Moreover, future research about the ADC reliability from multi-disciplinary perspectives is presented.

The research is sponsored by the projects (61105103, 51375059) supported by National Natural Science Foundation of China, and the project (3162021) supported by Beijing Natural Science Foundation.

The author declares there is no conflict of interest.

1. Adouane L. Toward fully autonomous vehicle navigation: From behavioral to hybrid multi-controller architectures.2017 11th International Workshop on Robot Motion and Control, IEEE; 2017:85–98.
2. Hanson JA, Bliss JP, Harden JW, et al. The effects of reliability and criticality on an IED interrogation task. InProceedings of the Human Factors and Ergonomics Society Annual Meeting. 2014;58(1):2340–2344.
3. Schwarting W, Alonso-Mora J, Rus D. Planning and decision-making for autonomous vehicles. Annual Review of Control, Robotics, and Autonomous Systems. 2018;1:187–210.
4. Duchoň F, Hubinský P, Hanzel J, et al. Intelligent vehicles as the robotic applications. Procedia Engineering. 2012;48:105–114.
5. Gomes L. Hidden Obstacles for Google’s Self-Driving Cars: Impressive Progress Hides Major Limitations of Google’s Quest for Automated Driving. Univ Parma, Italy, Tech Rep; 2014.
6. Blanco M, Atwood J, Russell S, Trimble T, McClafferty J, Perez M. Automated vehicle crash rate comparison using naturalistic data. Virginia Tech Transportation Institute; 2016.
7. Uber reportedly thinks its self-driving car killed someone because it ‘decided’ not to swerve.
8. Self-Driving Tesla Was Involved in Fatal Crash. US Says.
9. Belbachir A, Boutteau R, Merriaux P, et al. From autonomous robotics toward autonomous cars. In Intelligent Vehicles Symposium (IV), IEEE; 2013:1362–1367.
10. Fagnant DJ, Kockelman K. Preparing a nation for autonomous vehicles: opportunities, barriers and policy recommendations. Transportation Research Part A: Policy and Practice. 2015;77:167–181.
11. Koopman P, Wagner M. Autonomous vehicle safety: An interdisciplinary challenge. IEEE Intelligent Transportation Systems Magazine. 2017;9(1):90–96.
12. Merlet JP. Interval analysis and reliability in robotics. International Journal of Reliability and Safety. 2009;3(1-3):104–130.
13. Driankov D. Fuzzy logic techniques for autonomous vehicle navigation. Physica; 2013.
14. Cardarelli E, Sabattini L, Secchi C, et al. Cloud robotics paradigm for enhanced navigation of autonomous vehicles in real world industrial applications. InIntelligent Robots and Systems,2015 IEEE/RSJ International Conference; 2015 Sep 28; 2015:4518–4523.
15. Wei J, Snider JM, Kim J, et al. Towards a viable autonomous driving research platform. In Intelligent Vehicles Symposium (IV), IEEE; 2013:763–770.
16. Levinson J, Askeland J, Becker J, et al. Towards fully autonomous driving: Systems and algorithms. InIntelligent Vehicles Symposium (IV), IEEE; 2011:163–168.
17. Dhillon BS. Robot system reliability and safety: A Modern Approach. CRC Press; 2015.
18. Google self-driving car testing report on disengagements of autonomous mode. 2015:1–33.
19. Kalra N, Paddock SM. Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability? Transportation Research Part A: Policy and Practice. 2016;94:182–193.
20. Berger C, Rumpe B. Autonomous driving-5 years after the urban challenge: The anticipatory vehicle as a cyber-physical system. 2014.
21. Hojjati-Emami K, Dhillon B, Jenab K. Reliability prediction for the vehicles equipped with advanced driver assistance systems (ADAS) and passive safety systems (PSS). International Journal of Industrial Engineering Computations. 2012;3(5):731–742.
22. Teoh SS, Bräunl T. A reliability point and kalman filter-based vehicle tracking technique. In International Conference onIntelligent Systems. 2012:134–138.
23. Shi J, Meng Y, Wang S, et al. Reliability and safety analysis of redundant vehicle management computer system. Chinese Journal of Aeronautics. 2013;26(5):1290–302.
24. Broggi A, Medici P, Zani P, et al. Autonomous vehicles control in the VisLab intercontinental autonomous challenge. Annual Reviews in Control. 2012;36(1):161–171.
25. Nothdurft T, Hecker P, Frankiewicz T, et al. Reliable information aggregation and exchange for autonomous vehicles. InVehicular Technology Conference (VTC Fall), IEEE; 2011 Sep 5; 2011:1–5.
26. Kianfar R, Falcone P, Fredriksson J. Safety verification of automated driving systems. IEEE Intelligent Transportation Systems Magazine. 2013;5(4):73–86.
27. Aeberhard M, Rauch S, Bahram M, et al. Experience, results and lessons learned from automated driving on Germany's highways. IEEE Intelligent Transportation Systems Magazine. 2015;7(1):42–57.
28. Milella A, Reina G. 3D reconstruction and classification of natural environments by an autonomous vehicle using multi-baseline stereo. Intelligent Service Robotics. 2014;7(2):79–92.
29. Fernandes P, Nunes U.Platooning with DSRC-based IVC-enabled autonomous vehicles: Adding infrared communications for IVC reliability improvement. InIntelligent Vehicles Symposium (IV), IEEE; 2012:517–522.
30. Hardy J, Campbell M. Clustering obstacle predictions to improve contingency planning for autonomous road vehicles in congested environments. InIntelligent Robots and Systems (IROS), IEEE/RSJ International Conference; 2011 Sep 25; p. 1605–1611.
31. Broggi A, Cerri P, Ghidoni S, et al. A new approach to urban pedestrian detection for automatic braking. IEEE Transactions on Intelligent Transportation Systems. 2009;10(4):594–605.
32. Li Y, Cho JJ, Ren Y. How Can the Petroleum Industry Benefit From Human Reliability Analysis? InIADC/SPE Drilling Conference and Exhibition; Society of Petroleum Engineers; 2014.