A safety culture survey provides a huge amount of information for developing patient safety improvement programs in healthcare organizations and even the healthcare industry itself. Yet it is very difficult to engage healthcare professionals in participating in such a survey. Although many reasons may explain the low response rate, the most important factor might be how much of a burden the respondents experience when completing the survey. We hypothesized that the different levels of measurement scale influence the time to complete the survey and compared the time required to complete the questionnaire using different response formats: 5-point Likert, 3-point Likert, and simple disagree/agree scales. A 3-point scale showed a 9% decrease in completion time while the disagree/agree scale showed a 18.4% decrease in completion time compared to that of the currently used 5-point Likert scale. Interestingly, non-clinical personnel spent 60.7% more time completing the questionnaire than physicians, although no significant difference emerged among different clinical job types. The result of this study cannot and should not be the sole rationale for changing the measurement scales of existing instruments, but its results suggest that we should consider such alternative options and conduct more sophisticated studies. This study is an empirical supplement to our previous study, which examined the much more theoretical and psychometric parts of measurement scale issues. Thus, we recommend readers review it to get a complete picture of this endeavor to develop a more efficient patient safety culture measurement instrument.

Keywords: patient safety, safety culture, safety attitudes questionnaire, measurement scale

In our recent article, we showed two different approaches to providing more efficiency in administering the Safety Attitudes Questionnaire (SAQ): One was to reveal items conveying a small amount of information so that they could be removed from the survey questionnaire without compromising its validity;¹ the other was to adopt different levels of measurement, such as a 3-point Likert scale and simple agree/disagree (dichotomized) scale. The former approach does not need any more explanation, but the latter requires empirical evidence. The different response options provide valid estimates of safety culture compared to the previously used 5-point Likert scale. With the help of item response theory, we carefully reached the tentative conclusion that a simpler scale can provide almost equivalent culture estimates to the original scale.² However, for the second approach of using simpler response options, a missing link remained—namely, whether such collapsed response options really reduce the time required to complete the survey. Without such an understanding, any discussion on efficiency of different response options is meaningless. To answer the question, we conducted this simple study; we applied different response options to SAQ and measured the time the participating healthcare professionals needed to complete the questionnaire. We are aware of the issues caused by measurement scale changes, especially qualitative aspects such as how people react differently when faced with different response options. Most of those issues were addressed in our previous article.² Thus, we intentionally neglected those theoretical issues and focused exclusively on the time required to finish the survey when using different response options: a 5-point Likert scale (1=disagree strongly, 2=disagree slightly, 3=neutral, 4=agree slightly, 5=agree strongly), a 3-point Likert scale (1=disagree, 2=neutral, 3=agree), and a dichotomized scale (0=disagree, 1=agree).

We developed three sets of 34-item SAQ Korean versions (SAQ-K) with the previously mentioned different response options. We then recruited 104 healthcare professionals working in four different tertiary care hospitals. Each of these professionals recruited two more people working in the same clinical area and sharing similar characteristics (i.e., job type, work experience, and gender). Thus, we had 104 groups, each of which consisted of three healthcare professionals with similar characteristics, resulting in 312 participants in total. This clustering of participants into 104 groups was intentional. As each clinical area has a very different environment in terms of workload, hierarchical structure, and so forth, we tried to control this variation to a certain degree using this semi-matching manner.

The three questionnaire versions with different response options were randomly distributed in each group; one completed the 5-point scale version, another the 3-point version, and another the disagree/agree version. Just before the main question items began, respondents were asked to record the current time to the second; at the end of the items, the current time was asked again. Using these responses, we calculated the time required to complete the survey.

Collected data were entered into the computer and analyzed. Observations with too short or too long times, which were practically invalid, were removed from the analysis, leaving a total of 304 completed questionnaires. Because the outcome variable of our interest, time, was heavily right skewed (long tail on right side), which was eventually not controlled for by participants’ characteristics, we log-transformed the time variable.  

We next applied the generalized estimating equation (GEE), taking into account the clustering effect of groups—correlation among each triplet. We entered participants’ characteristics as categorical variables in the model to control their effects. Robust standard errors were estimated to prevent wrongful inference. Regression diagnostics were checked.

For all analyses, Stata 14.2 (StataCorp, College Station, Texas) was used.

Table 1 shows the participants’ characteristics. Each scale was meant to have the same number of healthcare professionals, but removing outliers caused small differences in the number of respondents across scales. The final number of participants for each scale was 101, 103, and 100 for the 5-point, 3-point, and disagree/agree scales, respectively, allowing for quite stable comparisons. In terms of job types, the majority of participants (N=185; 60.9%) were nurses, corresponding well to the human resource composition of Korea’s healthcare situation. They were followed by physicians (N=45; 14.8%) and medical technicians (e.g., radiological technicians or physical therapists) with 35 people, comprising 11.5%. In addition, 39 non-clinical personnel (e.g., administrative staff) participated, accounting for 12.8% of the respondents. With regard to work experience, 110 respondents (36.2%) had 1–5 years of experience, followed by 77 people (25.3%) with 11–20 years of experience, 45 people (14.8%) with 6–10 years of experience, and 36 people (11.8%) each with less than 1 year or more than 20 years of experience. Most nurses are female in Korea, which was reflected in the respondents’ gender: 218 (71.7%) were female while 86 (28.3%) were male.

Figure 1, the kernel density plot of time, shows the overall distribution of the time required to complete the survey for all participants. Those who misspecified the duration as being less than 1 minute were removed; thus, the plot begins at around 60 seconds. The density peaked at just under 200 seconds and decreased beyond that, leaving a long tail on the right side. This is a typical right-skewed distribution, suggesting that ordinary normal distribution-based analysis might not work. Indeed, when we conducted the analysis, the model was quite unstable. Thus, we log-transformed the time variable and achieved a model effectively satisfying regression assumptions, such as homoscedasticity, although this is not described in detail here.  

Figure 1 Kernel density plot of time (second) to complete a questionnaire of respondents.

Table 2 shows the results of the regression analysis of the log-transformed time variable on different response scales and respondents’ characteristics. Regression coefficients, standard errors, and 95% confidence intervals (CI) are shown in exponentiated values. To help understand, we added a column, % difference, before the exponentiated regression coefficients exp (b).

Using the 5-point Likert scale as the baseline, the 3-point Likert scale required a 9% shorter time for a respondent to complete the survey, with marginal significance (p=0.086). The disagree/agree scale showed a huge difference, 18.4% less time, compared to the 5-point scale (p=.001).

With regard to job types, we set physicians as the baseline. Nurses and medical technicians required 15.3% and 8.1% more time than physicians, respectively, but these differences were far from statistically significant. Non-clinical employees showed a huge difference, requiring 60.7% more time than physicians (p=0.000). In terms of work experience, we used healthcare workers with less than 1 year of experience as the baseline. All groups required more time than the baseline group, especially the 11–20 years group, which required 29.6% more time (p=0.026). Differences between males and females were negligible.

Time matters to everybody, yet a hospital is probably the place where time matters most. Each and every second is used to save lives; thus, we should secure healthcare professionals’ time as much as possible as their time is directly related to patients’ lives.

Quite often, people in charge of the quality and safety of a healthcare organization grumble about healthcare professionals’ lower-than-expected participation in quality and safety improvement endeavors. A patient safety culture survey is one example of such seemingly ignored endeavors. Honestly, we too were some of those very grumblers. However, one day we questioned whether evidence existed to show that healthcare professionals taking the time to fill out this survey saved more lives than seeing more patients. From our extensive experience of using survey results in developing a safety improvement strategy, our answer is a resounding yes. Yet we were not sure what kinds of efforts we put into saving time for those participating in surveys.

We believe that it is our obligation to make the burden to participate in surveys as small as possible, and the burden might be symbolized by how much time healthcare professionals spend completing such surveys. Therefore, we need to develop a strategy to lessen the burden, instead of grumbling. We should understand that clinicians must run for a patient once a patient’s heart stops beating regardless of whether they have finished a survey or not. We should understand that their schedule is always packed. We probably failed to admit this, withdrawing ourselves into a persecutory delusion that our endeavor in safety was being ignored. Thus, we decided to overhaul our tools, and we showed that reducing response options can decrease a considerable amount of time in completing surveys. Yet, there are concerns about it of course, such as the decreased granularity of culture estimates. Those were described in our previous article in detail. Readers are encouraged to refer to it.²

An unexpected result that emerged is that nonclinical personnel required much more time to complete the survey. Although we do not want to hastily interpret this phenomenon, one potential explanation is that many items on the SAQ-K and the original SAQ were designed for clinical personnel (e.g., “Nurse input is well received in this clinical area”). Such questions are certainly not easy—and perhaps impossible—for administration staff and any other personnel not working in a clinical area to answer. Furthermore, we intentionally eliminated a “N/A (not applicable)” option from the response options, forcing respondents to answer all items in the questionnaire to better measure the time devoted exclusively to choosing answers. Thus, these respondents might have suffered from having to choose answers for many clinical area-specific items. Indeed, we do not even know how they were able to answer those questions. Maybe those items were part of the reason for the longer time required to complete the questionnaire. In any event, it is obvious that we should develop different versions of SAQs designed or at least modified for specific areas or job types. Again, such strategies to improve SAQ were described in our previous articles.^1,2 

This study was conducted only as a simple supplement to our previous article, providing some empirical evidence on the effect of measurement scale choice on the time to complete a patient safety culture survey (SAQ-K). Although simple, we hope that this article, combined with our previous research, can encourage researchers to examine the efficiency issues of safety culture survey more thoroughly and reduce the burden on healthcare professionals. Without a doubt, we will be doing so too.

None.

None.

1. Jeong HJ, Lee WC. Item Response Theory-Based Evaluation of Psychometric Properties of the Safety Attitudes Questionnaire-Korean Version (SAQ-K). Biometrics & Biostatistics International Journal. 2016;3(5):1‒15.
2. Jeong HJ, Lee WC. The level of collapse we are allowed: Comparison of different response scales in Safety Attitudes Questionnaire. Biometrics & Biostatistics International Journal. 2016;4(4):1‒7.