Inconsistent work performance in automation, can we measure trust in automation?

doi:10.15406/iratj.2017.03.00075

eISSN: 2574-8092

International Robotics & Automation Journal

Opinion Volume 3 Issue 6

Inconsistent work performance in automation, can we measure trust in automation?

Heejin Jeong,¹ Jaehyun Park,² Jangwoon Park,³ Byung Cheol Lee³

Verify Captcha

Regret for the inconvenience: we are taking measures to prevent fraudulent form submissions by extractors and page crawlers. Please type the correct Captcha word to see email ID.

¹Department of Industrial and Operations Engineering, University of Michigan, USA
²Department of Industrial & Management Engineering, Incheon National University, South Korea
³Department of Engineering, Texas A&M University-Corpus Christi, USA

Correspondence: Byung Cheol Lee, Texas A&M University-Corpus Christi, USA, Tel 1(361) 825-4134

Received: October 28, 2017 | Published: December 12, 2017

Citation: Jeong H, Park J, Park J, et al. Inconsistent work performance in automation, can we measure trust in automation? Int Rob Auto J. 2017;3(6):378-379. DOI: 10.15406/iratj.2017.03.00075

Download PDF

Abstract

Automation is prevalent in modern workplaces, and its benefits have been proved in various domains. However, the improved performance levels are not consistent, and trust in automation also is difficult to predict and assess. To achieve the optimal interaction between human and automation, this article looked at three major causes of inconsistent work performance in automation: individual difference, subjective self-reporting measuring, and indistinctive automation levels. This article suggests some aspects to consider in measuring trust in automation.

Keywords: automation, trust, work performance, individual difference

Introduction

As automation is widely adopted and used in the modern working environment, technology develops the interaction between human and automation has become increasingly important. When human-automation interaction performs smoothly, the system efficiency and work performance will improve. However, optimum interactions are not always achieved. Automated systems improve the quality of work performed and the safety of human operators by reducing the risk of errors and decrease the level of effort and stress during the operation. However, the benefits of automation may not always be realized, and work performance is inconsistently influenced by different types, levels, and degrees of automation.^1,2 In addition, an indiscrete and excessive dependence of automation, sometimes, allows easily involving secondary tasks and making to neglect monitoring and follow-up steps in a primary task.³ Furthermore, it may interfere with the skill acquisition without automation.

Individual difference and subjective approach to measure work performance

Inconsistent work performance in automation is mainly caused by individual difference and the misjudgment of the capability of automation. The former that has been discussed enough in a wide range of psychology, sociology, human factors, and human-computer interaction literature is mainly associated with personality and cognitive ability on human task performance. Due to variant cognitive and physical capabilities, even the same automation cannot be uniformly affected on task performance in different operators. Particularly, the information-processing ability and working-memory capacity, which are the crucial capabilities to form mental models to perform automated tasks, are widely varied, and the variance of these capabilities produces the different and irregular patterns of task performance and situation awareness by different individual profiles.⁴

Another aspect we need to address in individual difference is how to measure the performance. Task performance is difficult to be measured in a concrete way, and impartial, reliable measurement of human task performance is not easy. Though diverse approaches have been used, indirect subjective approach such as self-reported survey is the most frequent method. However, human tends to perceive and respond differently to the overall experience of whole task scenarios and instant reaction.⁵ Although the subjective performance measuring method is convenient, there are always some issues as to whether any form of self-report accurately reflects respondents' "true" perceptual experiences.⁶ To establish the validity of ratings of perceived performance, several studies suggested bringing such subjective ratings under experimental control by demonstrating the association with objective factors.^6,7 Depending on subjective self-report measures can be considered as a potential cause of inconsistent performance in automation.

Automation classification and unclear distinction

We also need to look at the cause of inconsistent performance from the view of automation side. Previous research has defined the levels of automation by the amount of automation autonomy and the amount of human activity.⁸ Automation also can be classified by the classic human-machine task allocation and the stage of information processing. Sheridan & Verplank⁹ suggested 10 levels of automation as the basis of the classic human-machine task allocation principle. Several studies proposed that automation could be classified according to the four-stage model of information processing.^2,8 This classification aids to formulate specific function allocation schemes for application for automation of dynamic control tasks. For each function in the model (or stage of information processing), either human or machine or some combination thereof was held responsible for the effects on task performance due to the automation.

Despite the classification, the automation effects on task performance may not be precisely described and discretely allocated into the level or stage of automation. Evidently, automation improves the performance, but the failure of automation negatively influences performance. The concept of “more automation” or “high degree of automation” can be represented by a higher level, by later stage, or by both Kaber, Omal & Endsley.¹⁰ Though, typically, we assume that these two dimensions increase the degree of automation, “more automation” cannot be determined by the degree of automation in higher level with earlier stage and lower level with later stage.¹¹ In addition, previous research claimed that each level or stage could be automated to varying degrees to define overall autonomy of the system.¹ The stages of automation are defined by the functions of the information processing, but the effects of automation are difficult to be assigned to each function. An appropriate degree of automation may not be defined by the levels or stages, and a single degree of automation can include various functions.⁹ In contrast, the optimal degree of automation for each function may be primarily determined by an iterative design process based on costs to human performance. Thus, task performance is improved by the degree of automation, but the levels or stages alone are not enough to define specific degrees that differentiate the task performance. This reasoning suggests that degree of automation cannot be simply determined and it significantly affects the task performance in different automation.

Conclusion

Both individual difference and the level of automation bring compounding effects on the task performance. If the automation levels are overly advanced, human operators will be easily bored and hard to be aroused when an emergency or the situation requiring manual controls occurs. On the other hand, if the automation is at a primitive level, they will tend to complete the tasks without the supports of automation. In addition, the cognitive complexity to complete the automated tasks can be changed with the additional automation.¹² That is, inconsistent effects of automation may reflect not only individual differences in information processing but also different types or levels of automation. Such inconsistent outcomes from automation sometimes cause catastrophic results in several disciplines. We need to identify the main patterns and causes of the inconsistent outcomes in human operators’ task performance in the view of the user-oriented and the system-oriented cause in human-automation interactions. Specifically, we also need to investigate if automation success and failure work differently influence and if more automation (higher level and later stage) is more beneficial to task performance.