The road not taken: new methods to describe safety culture survey score

doi:10.15406/bbij.2017.06.00166

eISSN: 2378-315X

Biometrics & Biostatistics International Journal

Editorial Volume 6 Issue 3

The road not taken: new methods to describe safety culture survey score

Heon Jae Jeong,¹ Wui Chiang Lee,²

Verify Captcha

Regret for the inconvenience: we are taking measures to prevent fraudulent form submissions by extractors and page crawlers. Please type the correct Captcha word to see email ID.

Hsun Hsiang Liao³

¹The Care Quality Research Group, Chuncheon, Korea
²Department of Medical Affairs and Planning, Taipei Veterans General Hospital & National Yang-Ming University School of Medicine, Taipei, Taiwan
³Deputy Executive Officer, Joint Commission of Taiwan, Taiwan

Correspondence: Wui-Chiang Lee, Department of Medical Affairs and Planning, Taipei Veterans General Hospital & National Yang-Ming University School of Medicine, Taipei, Taiwan, Tel 886-2-28757120, Fax 886-2-28757200

Received: July 25, 2017 | Published: July 31, 2017

Citation: Jeong HJ, Lee WC. The road not taken: new methods to describe safety culture survey score. Biom Biostat Int J. 2017;6(3):332-333. DOI: 10.15406/bbij.2017.06.00166

Download PDF

For the past couple of decades, the healthcare industry has sought to measure safety culture of healthcare professionals. Instruments that measure culture, using teamwork as a major domain, treat each clinical arena or hospital as the unit of analysis. Certainly, the indicators from these questionnaires help organizations decide the appropriate direction to follow. However, the traditionally used indicators derived from these questionnaires have obvious shortcomings. They almost always use a single score as the end product—namely, the mean score of a work area or hospital. Even when using the percent agreement, the output is eventually a single number. We became curious. Looking back at the moment when we first learned statistics, the first words we heard were probably mean and standard deviation, which always went hand in hand. Statisticians’ terminology includes central tendency and spread, depicting a distribution of the data on hand. As such, is a single score like mean or percent agreement really enough to describe the topography of safety culture? We do not think so.

Let us imagine an intensive care unit (ICU) at a hospital. Doctors believe that the entire ICU crew is a good team because their definition of team is that doctors order and nurses follow, period. Yet nurses who work in this hierarchical environment voice displeasure with doctors’ unnecessary bravado, rudeness, refusal to allow nurses to ask questions, and so on. We fully expect the teamwork score from the healthcare professionals in the ICU to vary widely, with doctors’ scores being located in the high value region while nurses’ scores are very low. However, the mean score would be somewhere around the middle, obviously deceiving us.

Now, let us imagine a different ICU where all healthcare professionals respond they have “relatively” good teamwork, and their mean score is similar to that of the abovementioned ICU. We are not comparing which ICU is better, but we can clearly say that the two ICUs are completely different animals with regard to teamwork climate. Therefore, the teamwork of the two ICUs should never be regarded as the same, which means different teamwork improvement approaches must be applied.

We have long reviewed several safety culture indicators, and our verdict is that any indicator or set of indicators should reflect both the central tendency and spread. We propose two ways to address this: a graphical description and a single composite measure including both the mean and standard deviation of safety culture scores. We named these approaches the Jeong & Lee Quadrant (Safety Culture Grid) and the Jeong & Lee Index, respectively (following the authors’ names). They are being tested in a variety of healthcare settings at the time of this writing, and the interim results are very promising.

Figure 1 provides an example of the Jeong & Lee Quadrant in a graphical display of safety culture (i.e., teamwork). Each dot stands for a clinical arena. Using the median values of the mean score and standard deviation of each area, we developed a quadrant. In this setting, the lower right quadrant, with high mean scores and a narrow variance, is the best-case scenario, depicting responses from most healthcare professionals that they are on a good team. The upper left quadrant has the exact opposite meaning. Of course, the y-axis can be reversed to make the upper right quadrant represent a good team environment, if readers are more accustomed to such a format.

Figure 1 Jeong & Lee quadrant (safety culture grid).

On the other hand, the Jeong & Lee Index, a single composite indicator, is somewhat easier for frontline safety managers to internalize, as it maintains one score format as the mean. However, the calculation is a bit different, as shown in the following formula for a clinical unit:

$J e o n g & L e e I n d e x o f a U n i t = \frac{M e a n S c o r e o f t h e R e s p o n d e n t s i n a U n i t}{S t a n d a r d D e v i a t i o n o f t h e R e s p o n d e n t s o f a U n i t}$

As someone with a strong statistical background might notice, this index is the reverse of the coefficient of variation (CV) that is widely used in the manufacturing industry or chemical labs. However, the CV’s focus is variation, not central tendency. In a multiple regression setting, it is similar to a standardized beta, but the purpose it is to make it possible to compare the effects of different concepts, such as age and sex, the original idea of which is not concordant to our new index.

Interpreting the Jeong & Lee Index is clear (refer to Figure 1). If five clinical areas show the same mean score (numerator), the best area would be the one with the smallest standard deviation (denominator). The other way around is also the same: If five units show the same denominator (standard deviation), then the mean score decides which area is considered better.

We admit that no indicators are perfect: Overly simple indicators naturally lose the real-world information whereas overly complex and numerous indicators lose their practicality in use. Thus, developing a patient safety culture indicator might be like walking a tightrope, continuously trying to balance between the amount of information and ease of use. Yet one thing is clear: Sticking to the incomplete traditional indices might simply be denying the call of duty. At the end of the day, we have to switch to a better one. We have already felt the need to fix the current indicators, and fixing and improving something is what we do every day. It is time to consider new indicators. The reservation of a decision is not allowed in the field we chose as lives must be saved.

We have introduced two innovative safety culture indicators, and we do hope this short article ignites active discussion on this issue. We will certainly keep doing our part.