Submit manuscript...
eISSN: 2378-315X

Biometrics & Biostatistics International Journal

Correspondence:

Received: January 01, 1970 | Published: ,

Citation: DOI:

Download PDF

Abstract

This simple article is an addendum to our previous study to handle under-reporting issues in medical error reporting systems. If you are interested in the qualitative aspect of the reported errors, you may want to refer to it. If your interest resides more in the comparing of the numbers of error reports for various situations, while controlling for the under-reporting problem, this article offers you some hints. We used computerized prescriber order entry (CPOE) as an example, comparing the difference in prescribing error reports between hospitals with and without CPOE. After controlling all potential covariates, including hospital-level clustering, we found hospitals with CPOE experienced 2.48 times more prescribing error reports than hospitals without CPOE. The result itself is meaningful, but we suggest you focus more on the methodology, which is our contribution to the patient safety society to overcome underreporting issues with voluntary error reporting system data.

Keywords: voluntary error report data, computerized prescriber order entry, CPOE, under-reporting, reporting and learning systems

Introducton

Underreporting has been a serious problem for patient safety researchers and practitioners who want to analyze medical error reports data. As we never know the tendency of an individual healthcare professional or an organization as a whole in reporting the errors detected,14 any cross-sectional or longitudinal studies ought to be contaminated, although the amount of bias is uncertain. Jeong et al. developed a revolutionary methodology to indirectly control this underreporting phenomenon.1,4,5 Figure 1 helps explain the underreporting phenomenon, which is denoted as the difference between detected and reported errors.

In our previous study, we examined how a counter-error measure, computerized prescriber order entry (CPOE), catches an error before reaching the patient.1 In other words, we measured the CPOE effectiveness by comparing patterns of severity of reported errors between hospitals with and without CPOE. Although it worked well, that approach naturally led to an important question: If CPOE is so effective in detecting errors, are there any increases in the number of detected errors in hospitals where CPOE is in use compared to hospitals without CPOE? It is indeed a valid question.

Figure1 Relationship between detected errors and reported errors.6

This article answers this question using the same dataset as in our previous article. Thus, we strongly recommend you read the previous article or at least these two articles side by side.

Before we move on, let us make clear the modus operandi of CPOE: CPOE improves medication safety by making errors easily detectable, utilizing standardized data entry forms, and more importantly eliminating issues that arise from handwritten orders. As CPOE is designed for the prescribing phase of the medication use process, the administering phase gets a minimum impact from CPOE, as proven in our previous study (Figure 2).6

Figure 2 Medication use process, modified from “Preventing Medical Errors”.6

As such, we devised a methodology to catch the changes in the number of error reports from prescribing phase due to the implementation of CPOE, while controlling for the underreporting issue, the impact of which varies across hospitals. Let us directly dive into the methodology.

Methods

Data Source

The data used in this study were collected by MEDMARX, a national voluntary medication error reporting system in the US that was developed and operated by US Pharmacopeia.4 The dataset included medication error reports since 2003, the first year when MEDMARX collected the CPOE use status of each participating hospital, through 2007. To make the study stable, we narrowed the scope down to the inpatient setting, where the above-mentioned medication process (Figure 2) occurs in the hospital; thus, any upstream errors can be tracked and recorded. Note that the finest resolution of MEDMARX data is year, so any analysis with finer granulation was not an option.

Analysis

First, what we want to examine is the change due to CPOE in the error detection rate, which can be shown in the form of a ratio of two rates. For statistical purposes, we can refer to this as a rate ratio or incidence-rate ratio (IRR). The IRR of detected errors between the situation in which a hospital uses CPOE (i') and the situation where a hospital does not (i) can be denoted as:

Incidence Rate Ratio= Rate of Detected Errors at Hospital i' Rate of Detected Errors at Hospital i MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qacaWGjbGaamOBaiaadogacaWGPbGaamizaiaadwgacaWGUbGa am4yaiaadwgacaGGGcGaamOuaiaadggacaWG0bGaamyzaiaacckaca WGsbGaamyyaiaadshacaWGPbGaam4Baiabg2da9maalaaapaqaa8qa caWGsbGaamyyaiaadshacaWGLbGaaiiOaiaad+gacaWGMbGaaiiOai aadseacaWGLbGaamiDaiaadwgacaWGJbGaamiDaiaadwgacaWGKbGa aiiOaiaadweacaWGYbGaamOCaiaad+gacaWGYbGaam4Caiaacckaca WGHbGaamiDaiaacckacaWGibGaam4BaiaadohacaWGWbGaamyAaiaa dshacaWGHbGaamiBaiaacckacaWGPbGaai4jaaWdaeaapeGaamOuai aadggacaWG0bGaamyzaiaacckacaWGVbGaamOzaiaacckacaWGebGa amyzaiaadshacaWGLbGaam4yaiaadshacaWGLbGaamizaiaacckaca WGfbGaamOCaiaadkhacaWGVbGaamOCaiaadohacaGGGcGaamyyaiaa dshacaGGGcGaamisaiaad+gacaWGZbGaamiCaiaadMgacaWG0bGaam yyaiaadYgacaGGGcGaamyAaaaaaaa@923D@

The rate has a count as the numerator and time as the denominator. As such, as long as two rates have the same duration of time in the denominators (e.g., one year), counts can be utilized to calculate the rate ratio without considering the issue of offset or exposure.

Because the data that we have are reported data, the actual IRR that will be examined is the IRR of counts of reported errors. The discrepancy between detection and reporting will be addressed, and detailed mathematical formulae will be presented below.

To calculate IRR, Poisson models will be utilized to analyze the counts data.7 It is assumed that in a specific time interval, t, medication errors occur independently. The number of error reports y in a time interval t thus follows a Poisson distribution.

Pr( y|μ )=  exp( μ ) μ y y! MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qaciGGqbGaaiOCamaabmaapaqaa8qacaWG5bGaaeiFaiabeY7a TbGaayjkaiaawMcaaiabg2da9iaacckadaWcaaWdaeaapeGaaeyzai aabIhacaqGWbWaaeWaa8aabaWdbiabgkHiTiabeY7aTbGaayjkaiaa wMcaaiabeY7aT9aadaahaaqabeaajugWa8qacaWG5baaaaqcfa4dae aapeGaamyEaiaacgcaaaaaaa@4DB9@

where μ is the expectation of y and is depicted as

μ= λt MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qacqaH8oqBcqGH9aqpcaGGGcGaeq4UdWMaamiDaaaa@3D32@           (8)

Counts (number) of prescribing error reports per year were observed for different hospitals i with different characteristics (covariates). The mean count of prescribing error μ i MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qacqaH8oqBl8aadaWgaaqcfayaaKqzadWdbiaadMgaaKqba+aa beaaaaa@3BED@ is described with a log-linear model.8 In other words, if we have one covariate denoting whether a hospital has CPOE in use or not (0: CPOE is not in use, 1: CPOE is in use), x i MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qacaWG4bWcpaWaaSbaaKqbagaajugWa8qacaWGPbaajuaGpaqa baaaaa@3B34@ ,

ln( μ i )=  β 0 +  β 1 x i MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qaciGGSbGaaiOBaiaacIcacqaH8oqBl8aadaWgaaqcfayaaKqz adWdbiaadMgaaKqba+aabeaapeGaaiykaiabg2da9iaacckacqaHYo Gyl8aadaWgaaqcfayaaKqzadWdbiaaicdaaKqba+aabeaapeGaey4k aSIaaiiOaiabek7aITWdamaaBaaajuaGbaqcLbmapeGaaGymaaqcfa 4daeqaa8qacaWG4bWcpaWaaSbaaKqbagaajugWa8qacaWGPbaajuaG paqabaaaaa@5218@

From the above equation, by applying μ i =  λ i t MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qacqaH8oqBl8aadaWgaaqcfayaaKqzadWdbiaadMgaaKqba+aa beaapeGaeyypa0JaaiiOaiabeU7aSTWdamaaBaaajuaGbaqcLbmape GaamyAaaqcfa4daeqaa8qacaWG0baaaa@4476@ , we can obtain

λ i t=exp( β 0 + β 1 x i )=exp( β 0 )exp( β 1 x i ) MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qacqaH7oaBl8aadaWgaaqcfayaaKqzadWdbiaadMgaaKqba+aa beaapeGaamiDaiabg2da9iGacwgacaGG4bGaaiiCamaabmaapaqaa8 qacqaHYoGyl8aadaWgaaqcfayaaKqzadWdbiaaicdaaKqba+aabeaa peGaey4kaSIaeqOSdi2cpaWaaSbaaKqbagaajugWa8qacaaIXaaaju aGpaqabaWdbiaadIhal8aadaWgaaqcfayaaKqzadWdbiaadMgaaKqb a+aabeaaa8qacaGLOaGaayzkaaGaeyypa0JaciyzaiaacIhacaGGWb WaaeWaa8aabaWdbiabek7aI9aadaWgaaqaaKqzadWdbiaaicdaaKqb a+aabeaaa8qacaGLOaGaayzkaaGaaeyzaiaabIhacaqGWbWaaeWaa8 aabaWdbiabek7aITWdamaaBaaajuaGbaqcLbmapeGaaGymaaqcfa4d aeqaa8qacaWG4bWcpaWaaSbaaKqbagaajugWa8qacaWGPbaajuaGpa qabaaapeGaayjkaiaawMcaaaaa@6A49@

The ratio of rates of two organizations in one year, t, (or two different years of one organization) can be expressed as the ratio of two lambdas. To illustrate, λ i MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qacqaH7oaBl8aadaWgaaqcfayaaKqzadWdbiaadMgaaKqba+aa beaaaaa@3BEB@ and λ i' MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qacqaH7oaBl8aadaWgaaqcfayaaKqzadWdbiaadMgacaGGNaaa juaGpaqabaaaaa@3C96@ are used for the rates of two organizations i and i’, respectively.

λ i λ i' = λ i t λ i' t = exp( β 0 )exp( β 1 x i ) exp( β 0 )exp( β 1 x i ) =exp{ β 1 ( x i x i ) }=exp ( β 1 ) ( x i x i ) MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qadaWcaaWdaeaapeGaeq4UdW2cpaWaaSbaaKqbagaajugWa8qa caWGPbaajuaGpaqabaaabaWdbiabeU7aSTWdamaaBaaajuaGbaqcLb mapeGaamyAaiaacEcaaKqba+aabeaaaaWdbiabg2da9maalaaapaqa a8qacqaH7oaBl8aadaWgaaqcfayaaKqzadWdbiaadMgaaKqba+aabe aapeGaamiDaaWdaeaapeGaeq4UdW2cpaWaaSbaaKqbagaajugWa8qa caWGPbGaai4jaaqcfa4daeqaa8qacaWG0baaaiabg2da9maalaaapa qaa8qaciGGLbGaaiiEaiaacchadaqadaWdaeaapeGaeqOSdi2cpaWa aSbaaKqbagaajugWa8qacaaIWaaajuaGpaqabaaapeGaayjkaiaawM caaiaabwgacaqG4bGaaeiCamaabmaapaqaa8qacqaHYoGyl8aadaWg aaqcfayaaKqzadWdbiaaigdaaKqba+aabeaapeGaamiEaSWdamaaBa aajuaGbaqcLbmapeGaamyAaaqcfa4daeqaaaWdbiaawIcacaGLPaaa a8aabaWdbiGacwgacaGG4bGaaiiCamaabmaapaqaa8qacqaHYoGyl8 aadaWgaaqcfayaaKqzadWdbiaaicdaaKqba+aabeaaa8qacaGLOaGa ayzkaaGaaeyzaiaabIhacaqGWbWaaeWaa8aabaWdbiabek7aITWdam aaBaaajuaGbaqcLbmapeGaaGymaaqcfa4daeqaa8qacaWG4bWcpaWa aSbaaKqbagaajugWa8qaceWGPbWdayaafaaajuaGbeaaa8qacaGLOa Gaayzkaaaaaiabg2da9iGacwgacaGG4bGaaiiCamaacmaapaqaa8qa cqaHYoGyl8aadaWgaaqcfayaaKqzadWdbiaaigdaaKqba+aabeaape WaaeWaa8aabaWdbiaadIhal8aadaWgaaqcfayaaKqzadWdbiaadMga aKqba+aabeaapeGaeyOeI0IaamiEaSWdamaaBaaajuaGbaqcLbmape GabmyAa8aagaqbaaqcfayabaaapeGaayjkaiaawMcaaaGaay5Eaiaa w2haaiabg2da9iaabwgacaqG4bGaaeiCamaabmaapaqaa8qacqaHYo Gyl8aadaWgaaqcfayaaKqzadWdbiaaigdaaKqba+aabeaaa8qacaGL OaGaayzkaaWdamaaCaaabeqaa8qadaqadaWdaeaapeGaamiEaSWdam aaBaaajuaGbaqcLbmapeGaamyAaaqcfa4daeqaa8qacqGHsislcaWG 4bWdamaaBaaabaqcLbmapeGabmyAa8aagaqbaaqcfayabaaapeGaay jkaiaawMcaaaaaaaa@AEE1@

Now that counts for all hospitals have the same time frame, one year, the exponentiated coefficient exp( β 1 ) MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qacaqGLbGaaeiEaiaabchadaqadaWdaeaapeGaeqOSdi2cpaWa aSbaaKqbagaajugWa8qacaaIXaaajuaGpaqabaaapeGaayjkaiaawM caaaaa@4033@ is the IRR for a unit increase in x i MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qacaWG4bWcpaWaaSbaaKqbagaajugWa8qacaWGPbaajuaGpaqa baaaaa@3B34@ , indicating the IRR of prescribing errors between a hospital with CPOE and a hospital without CPOE.8

By adding more covariates, we can develop a more sophisticated model. The expected number of prescribing errors μ ij MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qacqaH8oqBl8aadaWgaaqcfayaaKqzadWdbiaadMgacaWGQbaa juaGpaqabaaaaa@3CDC@ reported in year ifor hospital j is described as the following log-linear model:

ln( μ ij )= β 0 + β 1 x 1ij +z ' ij γ MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qaciGGSbGaaiOBamaabmaapaqaa8qacqaH8oqBl8aadaWgaaqc fayaaKqzadWdbiaadMgacaWGQbaajuaGpaqabaaapeGaayjkaiaawM caaiabg2da9iabek7aI9aadaWgaaqaaKqzadWdbiaaicdaaKqba+aa beaapeGaey4kaSIaeqOSdi2cpaWaaSbaaKqbagaajugWa8qacaaIXa aajuaGpaqabaWdbiaadIhal8aadaWgaaqcfayaaKqzadWdbiaaigda caWGPbGaamOAaaqcfa4daeqaa8qacqGHRaWkcaWG6bqcLbmacaGGNa WcpaWaaSbaaKqbagaajugWa8qacaWGPbGaamOAaaqcfa4daeqaa8qa cqaHZoWzaaa@5C21@

where β 1 MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qacqaHYoGyl8aadaWgaaqcfayaaKqzadWdbiaaigdaaKqba+aa beaaaaa@3BA5@ means the log incidence-rate ratio between a hospital with CPOE and without CPOE and x 1ij MathType@MTEF@5@5@+= feaagKart1ev2aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qacaWG4bWcpaWaaSbaaKqbagaajugWa8qacaaIXaGaamyAaiaa dQgaaKqba+aabeaaaaa@3CDD@  indicates whether the hospital j has CPOE in use in the year i. The vector z ' ij MathType@MTEF@5@5@+= feaagKart1ev2aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qacaWG6bqcLbmacaGGNaWcpaWaaSbaaKqbagaajugWa8qacaWG PbGaamOAaaqcfa4daeqaaaaa@3DFD@ stands for adjustment variables, such as all other hospital characteristics for the year i for the hospital j, and the year the data are reported. These adjustment variables are controlled for in the regression model.9 These characteristics include the use of computer-generated medication administration records (MARs), inpatient pharmacy availability, average monthly medication doses, and so forth. The list of these characteristics is shown in Table 3. One covariate not yet described is the count of reported errors from the administering node. Assuming that implementing CPOE has a minimal impact on the administering node, the count of reported errors from the administering phase, along with the variable for how medication errors were identified, can be effectively used as an adjusting variable for the impact of an organization’s safety culture or propensity related to the reporting of detected medication errors.

CPOE use

Node

Prescribing

Transcribing

Administering

Total

For all clinical areas

 

 

 

 

                 n

9,227

2,836

6,569

18,632

row %

49.50%

15.20%

35.30%

100.00%

column %

71.20%

24.70%

25.90%

37.40%

Not in use

                 n

3,734

8,626

18,814

31,174

row %

12.00%

27.70%

60.40%

100.00%

  column %

28.80%

75.30%

74.10%

62.60%

Total

                 n

12,961

11,462

25,383

49,806

row %

26.00%

23.00%

51.00%

100.00%

  column %

100.00%

100.00%

100.00%

100.00%

Table 3 Number of error reports in each phase of the medication use process1

Although not shown in the previous equation, the impact of calendar year and other hospital characteristics, such as detailed facility owner information, is addressed in the results section.

For ordinary Poisson models, the independence of each count is assumed. However, in this particular analysis, we must address the fact that a hospital submitted error reports for multiple years. Therefore, counts of prescribing errors in a hospital for one year cannot be independent from those of the same hospital in a different year. The method for addressing the dependence of observations within a single organization is applying a random effects model including a hospital-specific intercept ζ 1j MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qacqaH2oGEl8aadaWgaaqcfayaaKqzadWdbiaaigdacaWGQbaa juaGpaqabaaaaa@3CB0@ :8

ln( μ ij )= β 0 + β 1 x 1ij +z ' ij γ+ ζ 1j MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qaciGGSbGaaiOBamaabmaapaqaa8qacqaH8oqBpaWaaSbaaeaa jugWa8qacaWGPbGaamOAaaqcfa4daeqaaaWdbiaawIcacaGLPaaacq GH9aqpcqaHYoGyl8aadaWgaaqcfayaaKqzadWdbiaaicdaaKqba+aa beaapeGaey4kaSIaeqOSdi2cpaWaaSbaaKqbagaajugWa8qacaaIXa aajuaGpaqabaWdbiaadIhal8aadaWgaaqcfayaaKqzadWdbiaaigda caWGPbGaamOAaaqcfa4daeqaa8qacqGHRaWkcaWG6bqcLbmacaGGNa WcpaWaaSbaaKqbagaajugWa8qacaWGPbGaamOAaaqcfa4daeqaa8qa cqaHZoWzcqGHRaWkcqaH2oGEl8aadaWgaaqcfayaaKqzadWdbiaaig dacaWGQbaajuaGpaqabaaaaa@630E@

μ ij E( y ij | x 1ij , z ij , ζ 1j )=exp( β 0 + β 1 x 1ij + z ij γ+ ζ 1j ) MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qacqaH8oqBl8aadaWgaaqcfayaaKqzadWdbiaadMgacaWGQbaa juaGpaqabaWdbiabggMi6kaadweadaqadaWdaeaapeGaamyEa8aada WgaaqaaKqzadWdbiaadMgacaWGQbaajuaGpaqabaWdbiaabYhacaWG 4bWcpaWaaSbaaKqbagaajugWa8qacaaIXaGaamyAaiaadQgaaKqba+ aabeaapeGaaiilaiqadQhapaGbauaalmaaBaaajuaGbaqcLbmapeGa amyAaiaadQgaaKqba+aabeaapeGaaiilaiabeA7a69aadaWgaaqaaK qzadWdbiaaigdacaWGQbaajuaGpaqabaaapeGaayjkaiaawMcaaiab g2da9iGacwgacaGG4bGaaiiCamaabmaapaqaa8qacqaHYoGyl8aada WgaaqcfayaaKqzadWdbiaaicdaaKqba+aabeaapeGaey4kaSIaeqOS di2cpaWaaSbaaKqbagaajugWa8qacaaIXaaajuaGpaqabaWdbiaadI hal8aadaWgaaqcfayaaKqzadWdbiaaigdacaWGPbGaamOAaaqcfa4d aeqaa8qacqGHRaWkceWG6bWdayaafaWcdaWgaaqcfayaaKqzadWdbi aadMgacaWGQbaajuaGpaqabaWdbiabeo7aNjabgUcaRiabeA7a6TWd amaaBaaajuaGbaqcLbmapeGaaGymaiaadQgaaKqba+aabeaaa8qaca GLOaGaayzkaaaaaa@7FD8@

=exp( ζ 1j )exp( β 0 + β 1 x 1ij + z ij γ ) MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qacqGH9aqpciGGLbGaaiiEaiaacchadaqadaWdaeaapeGaeqOT dO3cpaWaaSbaaKqbagaajugWa8qacaaIXaGaamOAaaqcfa4daeqaaa WdbiaawIcacaGLPaaaciGGLbGaaiiEaiaacchadaqadaWdaeaapeGa eqOSdi2cpaWaaSbaaKqbagaajugWa8qacaaIWaaajuaGpaqabaWdbi abgUcaRiabek7aI9aadaWgaaqaaKqzadWdbiaaigdaaKqba+aabeaa peGaamiEaSWdamaaBaaajuaGbaqcLbmapeGaaGymaiaadMgacaWGQb aajuaGpaqabaWdbiabgUcaRiqadQhapaGbauaalmaaBaaajuaGbaqc LbmapeGaamyAaiaadQgaaKqba+aabeaapeGaeq4SdCgacaGLOaGaay zkaaaaaa@5FA2@

where the conditional distribution of y ij MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qacaWG5bWdamaaBaaabaqcLbmapeGaamyAaiaadQgaaKqba+aa beaaaaa@3B8B@ given x 1ij , z ij  and   ζ 1j MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qacaWG4bWcpaWaaSbaaKqbagaajugWa8qacaaIXaGaamyAaiaa dQgaaKqba+aabeaapeGaaiilaiqadQhapaGbauaalmaaBaaajuaGba qcLbmapeGaamyAaiaadQgaaKqba+aabeaapeGaaeiOaiaabggacaqG UbGaaeizaiaabckacaGGGcGaeqOTdO3damaaBaaabaqcLbmapeGaaG ymaiaadQgaaKqba+aabeaaaaa@4ED2@ is a Poisson distribution and ζ 1j MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qacqaH2oGEl8aadaWgaaqcfayaaKqzadWdbiaaigdacaWGQbaa juaGpaqabaaaaa@3CB0@ is assumed to be independent and to follow a gamma distribution.9 This means that the counts of prescribing error reports for a facility j at different occasions are specified as conditionally independent on the given random intercept ζ 1j MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qacqaH2oGEl8aadaWgaaqcfayaaKqzadWdbiaaigdacaWGQbaa juaGpaqabaaaaa@3CB0@ and the covariates.8 The random intercept, ζ 1j MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qacqaH2oGEl8aadaWgaaqcfayaaKqzadWdbiaaigdacaWGQbaa juaGpaqabaaaaa@3CB0@ , can be thought of as the combined effect of any omitted facility-specific covariates that could lead some hospitals to have a higher propensity to submit more medication error reports than others.8

Therefore, the exponentiated estimate for β 1 MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqcfaieaaaaaa aaa8qacqaHYoGyl8aadaWgaaqcfayaaKqzadWdbiaaigdaaKqba+aa beaaaaa@3BA5@ from the above random effects Poisson model denotes the IRR of prescribing errors between a hospital with CPOE and a hospital without CPOE, after controlling for the under-reporting problem.

Results

Table 3 shows the total number of error reports from hospitals with CPOE and without CPOE broken down into four major phases of the medication use process. Please refer to the row percent. In hospitals with CPOE, 49.5% of error reports were from the prescribing phase; meanwhile, in hospitals without CPOE, only 12.0% of reports came from the prescribing phase. This table provides only the rough topography of error reports across various medication use phases, although it is still helpful. Let us move on to the main dish, Table 4.

Category

IRR

z

p-value

95% CI

CPOE for all clinical areas

2.48

5.71

0

1.81

3.38

Count of administering error reports

1.01

22.01

0

1.01

1.01

Computer-generated MAR's

On demand

0.71

-0.65

0.51

0.25

1.98

both batch and on demand

1.35

-0.9

0.37

0.7

2.58

No MAR's

1.75

1.5

0.13

0.84

3.63

Owner/Operator

 Government;Federal;VA

0.63

-1

0.32

0.26

1.56

 Government;Federal;Other

1.56

1.06

0.29

0.69

3.56

 Government nonfederal

0.46

-1.89

0.06

0.21

1.03

Pharmacist Availability

 On call when pharmacy closed

0.72

-1.18

0.24

0.42

1.25

  Not Available when pharmacy closed

0.06

-2.91

0

0.01

0.41

  Average Medication Doses (month)

10,000-19,999

0.5

-2.70

0.01

0.3

0.82

20,000-39,999

5.13

4.25

0

2.41

10.91

40,000-99,999

6.69

4.44

0

2.89

15.51

>=100,000

14.28

6.62

0

6.50

31.37

Calendar Year

2004

1.16

4.92

0

1.09

1.23

2005

1.54

12.59

0

1.44

1.65

2006

1.44

9.98

0

1.34

1.54

2007

1.38

8.46

0

1.28

1.49

Methods available for error detection

Staff initiated written reports

0.24

-4.7

0

0.13

0.43

Staff initiated electronic communication

0.79

-0.99

0.32

0.49

1.26

Automatic information system detection

0.65

-1.01

0.31

0.28

1.5

Random observation based reviews

1.29

0.9

0.37

0.74

2.25

Download from hospital IT/MR dept.

2.49

1.8

0.07

0.92

6.74

Telephone hot line

2.82

4.39

0

1.77

4.47

Patient or patient's family initiated

0.23

-6.27

0

0.15

0.37

Other

1.37

0.89

0.37

0.69

2.72

Constant

16.13

4.31

0

4.55

57.15

Random Effect

alpha

1.74

1.35

2.23

Table 4 Poisson regression result of number of prescribing error reports

Likelihood-ratio test of alpha = 0: Chibar2(01) = 13000 Prob >= chibar2 = 0.000

Reference category is government hospitals (federal military), the average monthly medication doses of which is less than 10,000, which have pharmacist 24 hours a day, 7 days a week and MARs only through batch processing, in the year of 2003.

Table 4 describes all the results from the full random effects Poisson regression model, on which we spent so many words to explain. The numbers are self-explanatory, so we focus only on the fact that the IRR for hospitals with CPOE was 2.48 (95% CI: 1.81–3.38), denoting that—when all other covariates including hospital-specific random effects were taken into consideration—hospitals where a CPOE system was in place submitted 2.48 times more prescribing errors than hospitals where a CPOE system was not in place.

Discussion

This is a methodology paper, so please refer to the discussion section of our previous article for a detailed discussion about the reporting system and its relationship with patient safety improvement.1 What we showed in this study is a suggestion of a methodology to utilize the number of error reports through voluntary medical error reporting systems as an indicator of patient safety-related situations of a healthcare organization or even beyond the individual organization level, such as a region or even a country. To our knowledge, the underreporting issue of the voluntary nature of such systems has always served as a sure hit from critics. We certainly know that there is no perfect way to control it, yet we do hope that this methodology along with the one that we introduced before1 helps our fellow safety researchers get the most out of their error reports that have long been waiting to be analyzed.

Acknowledgments

None.

Conflicts of interest

None.

References

Creative Commons Attribution License

© . This is an open access article distributed under the terms of the, which permits unrestricted use, distribution, and build upon your work non-commercially.