The capacity of electrocardiograms (ECG) recorded in hospital is increasing. Although modern electrocardiographs have digital output, most of ECG's are still recorded on grid papers. Recently, exchanging patients' clinical information between healthcare facilities has become very important. It will be very helpful if paper type of ECG could be converted to digital form. In this study, a morphological method is developed to digitize ECG signal from the paper charts. The results show that the method can erase the background noise and provide the digital ECG signal from ECG paper charts correctly. ECG signal digitization is practically used in ECG data exchanging between healthcare providers. Experimental results on sample ECG paper records are very encouraging and show the promise of efficiency in ECG data storage and retrieval and easy manipulation for clinical uses.

Keywords: paper record ECG, computer-based ECG, conversion techniques, image processing, biomedical signal processing

An ECG is simply a representation of the electrical activity of the heart muscle as it changes with time, usually printed on paper for easier analysis. Like other muscles, cardiac muscle contracts in response to electrical depolarization of the muscle cells. The sum of this electrical activity is amplified and recorded for just a few seconds that know as an ECG. The electrocardiogram (ECG) represents a graph of variation in electrical potential generated by the heart and recorded at the body surface. Recently, exchanging patients’ clinical information between healthcare facilities has become very important. Measurements of parameters from electrocardiograms (ECGs) are still largely performed from paper ECG records. It will be very helpful, if plane paper type of ECG could be converted to digital form. At the same time, the new requirements justify the need for tools to convert existing paper ECG records into digital format, particularly for retrospective studies. A computer application was developed for the conversion of paper ECG records to digital ECG files.^1-6 In general, the background of ECG paper charts can be divided into three types: uniform background, background with colored grid and background with black grid. Electrocardiogram (ECG) signals are usually recorded onto standard grid papers in hospitals as a routine clinical examination for diagnosis of possible cardiac failure. There is a need to convert the existing ECG paper records into electronic forms for efficient retrieval for clinical uses.

It is to be noted that printed ECG reports include some characters as well as annotations while taking an image of such ECG strips. It is desirable to remove these printed characters. Efficient noise removal and image enhancing algorithms are also required so as to increase overall digitization process accuracy.^7-10 This research work proposes an effective algorithm for extracting the digitized signal from ECG paper strips. A system prototype is designed to convert ECG paper records into electronic ECG recording forms so that they can be either efficiently retrieved as needed, or analyzed by ECG signal processing algorithms, or transmitted through computer networks for clinical purposes. In the research, the scanned binary images of ECG paper records are analyzed using image processing techniques, such as filtering and morphological procedures. The direction of digitizing the ECG paper records is referred in which morphological approach is used to remove the background grid and isolated pixels such as characters from the paper recorded ECG.

The ECG charts were scanned into images using a commercial A4 scanner with 600dpi gray scale settings. Before the operation, a region of interest is being defined that will help in reducing the processing time and much more it will allow a specific selection of a particular part of the waveform (Figure 1). The algorithm starts with reading an image and converts it from gray scale to binary image in order to prepare it for the binary morphological operation. A threshold is applied with image to binary conversion then the segments that are less than a specified number of pixels can be removed. The binary image is processed with a particular neighborhood morphological binary operation that will estimate the background of the image that will be subtracted from the selected image. The result image contains residual effects from the subtraction of the background. To reduce these effects, region based segmentation is applied. Then, the segments that represent the ECG wave and background are separated. Region based segmentation is applied for more isolated pixels that represent printed characters or noise reduction.^8-12 The final image will contain the waveform with minimal effect on the boundary. The next step is to smooth the boundary of the ECG wave with a binary morphological thicken. The final step is to convert the waveform in the final processed image to 1D signal that can be then manipulated for any further signal processing. This process is shown in the block diagram of (Figure 1).

Figure 1 Conversion of ECG from Paper Record to Computer-based ECG.

Digitization algorithm

ECG strips (printed on a thermal paper) are scanned and scanned images are enhanced for the signal lines sharper. The grid is removed from the scanned ECG images by using threshold and morphological operation. Region based segmentation is applied for removing isolated pixels that represent printed characters or noise.^13-15 Final processed ECG signal is obtained by interpolating the pixel values and the signal is represented by a specific signal pixel per column. The overall algorithmic steps are shown in Figure 2.

Figure 2 Flowchart of ECG Paper Record Digitization Process.

Digitization process

There are three main sections in ECG signal digitization process. They are as follow:

1. Background removal,
2. Characters and small object removal,
3. Conversion from waveform to 1-D signal.

Background removal

The following steps are implemented for the background grip removal.

1. Scan into images using a commercial A4 scanner with 600dpi gray scale settings
2. Start with reading an image.
3. Enhance Image.
4. Convert it from gray scale to binary image by applying a threshold level to remove the pixels that are less than a threshold value.
5. Process with a particular neighborhood morphological binary operation that will estimate the background of the image.

Scanning and standardization

Based on the resolution of scanning and paper speed, the pixel is defined in terms of time and amplitude units. For example, a resolution of 600 dpi implies 600 pixels in an inch (25.4 mm). Thus the number of pixels per mm can be calculated. Paper speed of 25 mm/sec i.e. 1 sec = 25 mm and calibration mark of 1mV amplitude = 10 mm is used to evaluate the value of each pixel in the time scale and amplitude scale. Pixel value in time scale is found to be 1.693 ms and amplitude scale to be 4.233 mV. These values are used during the digital time series signal generation.

Load image and enhancement

This step enhances the ECG image by making the signal lines sharper. Im adjust command is applied for making background noise lighter than the main ECG signal. J = im adjust (I) maps the intensity values in grayscale image I to new values in J such that 1% of data is saturated at low and high intensities of I. This increases the contrast of the output image J and adjusts image intensity values.

Convert gray scale to binary image based on threshold

This is followed by setting a threshold. Threshold value is chosen by comparing between noise pixels and pixels representing actual ECG signal. If ECG signal pixels values close to the threshold then pixels will be made darker by subtracting a fix value. The noise pixels values that close to the threshold value will be made lighter by adding a fix value. Therefore, the resulting image will contain distinct ECG signal in the image. BW = im2bw (I, level) converts the grayscale image I to a binary image. The output image BW replaces all pixels in the input image with luminance greater than level with the value 1 (white) and replaces all other pixels with the value 0 (black). Specify level in the range [0, 1]. This range is relative to the signal levels possible for the image's class. Therefore, a level value of 0.5 is midway between black and white, regardless of class. To compute the level argument, the function “gray thresh” can be used. If level is not specified, im2bw uses the value 0.5.

Characters and small object removal

After removing the background grip, the isolated pixels such as characters are still exist. To remove these small objects the following steps are applied

1. Local entropy of grayscale image,
2. Morphologically open binary image (remove small objects),
3. Region based segmentation to remove the isolated characters,
4. Remove the segments that are less than a specified number of pixels,
5. Subtract the ECG signal only (Remove characters),
6. Smooth the ECG signal by morphological thicken process because of containing residual effects from the subtraction of the background.

Local entropy of grayscale image

J = entropyfilt (I) returns the array J, where each output pixel contains the entropy value of the 9-by-9 neighborhood around the corresponding pixel in the input image I. “I” can have any dimension. If “I” has more than two dimensions, entropyfilt treats it as a multidimensional grayscale image and not as a true color (RGB) image. The output image J is the same size as the input image I. For pixels on the borders of I, entropyfilt uses symmetric padding. In symmetric padding, the values of padding pixels are a mirror reflection of the border pixels in I. J = entropyfilt (I, NHOOD) performs entropy filtering of the input image I where specify the neighborhood in NHOOD. NHOOD is a multidimensional array of zeros and ones where the nonzero elements specify the neighbors. NHOOD's size must be odd in each dimension. By default, entropyfilt uses the neighborhood true (9). Entropyfilt determines the center element of the neighborhood by floor ((size (NHOOD)+1)/2). The entropyfilt functions operate by defining a neighborhood around the pixel of interest and calculating the statistic for the neighborhood to determine the pixel value in the output image. The entropyfilt function calculates the entropy of the neighborhood and assigns that value to the output pixel. Note that, by default, the entropyfilt function defines a 9-by-9 neighborhood around the pixel of interest. To calculate the entropy of an entire image, use the following entropy function.

Entropy= - sum(pi*log2 pi)
Where pi is the probability that the difference between 2 adjacent pixels is equal to i, and Log2 is the base 2 logarithm.

In the above (Figure 3), the entropy value for the centre pixel “0” can be calculated as follow:
Probability of 1 = 3/9
Probability of 0 = 6/9
$\text{Entropy value for "0" }=\text{-}{\displaystyle \sum \left[\left(\frac{\text{3}}{\text{9}}{\log }_2\frac{3}{9}\right)+\left(\frac{6}{9}{\log }_2\frac{6}{9}\right)\right]}=0.918295$ For pixels on the borders of Image, entropyfilt uses symmetric padding. Therefore, the entropy value for the last pixel “0” in the above (Figure 4) can be calculated as follow. Take the four pixel value in the previous of “0” as a mirror reflection.
0  0  1  0  0  0  1 0  0
Probability of 1 = 2/9
Probability of 0 = 7/9

$\text{Entropy value for "0" that has last position}=\text{-}{\displaystyle \sum \left[\left(\frac{\text{2}}{\text{9}}{\log }_2\frac{2}{9}\right)+\left(\frac{7}{9}{\log }_2\frac{7}{9}\right)\right]}=0.7642$

The entropy value for the third pixel “0” in the above (Figure 5) can be calculated as follow. Take the interested pixel “0” as center and then fill the blank position of pixel as a mirror reflection.
1  0  1  1  0  0  0 0  1

Probability of 1 = 4/9
Probability of 0 = 5/9
$\text{Entropy value for "0" that has third position}=\text{-}{\displaystyle \sum \left[\left(\frac{\text{4}}{\text{9}}{\log }_2\frac{4}{9}\right)+\left(\frac{5}{9}{\log }_2\frac{5}{9}\right)\right]}=0.991076$
The entropy value for the other pixels can be calculated by using entropy function as described above.

Figure 3 Determining Pixel Values in Entropy Filtered Output Image (1).

Figure 4 Determining Pixel Values in Entropy Filtered Output Image (2).

Figure 5 Determining Pixel Values in Entropy Filtered Output Image (3).

Converts the matrix to the intensity image

I = mat2gray (A, [amin amax]) converts the matrix A to the intensity image I. The returned matrix I contains values in the range 0.0 (black) to 1.0 (full intensity or white). The amin and amax are the values in A that correspond to 0.0 and 1.0 in I.
I = mat2gray (A) sets the values of amin and amax to the minimum and maximum values in A. The input array A can be logical or numeric. The output image “I” is double.

Morphologically open binary image

BW2 = bw are a open (BW, P) removes from a binary image all connected components (objects) that have fewer than P pixels, producing another binary image, BW2. The default connectivity is 8 for two dimensions, 26 for three dimensions, and conndef (ndims (BW), 'maximal') for higher dimensions. In this research work, “bwareopen” is used for removing the isolated characters that pixels are connected continuously.

This step removes the isolated pixels that do not represent the signal (printed characters). This step comprises of the following steps:-

1. Step 0: Remove the frame of the image that is border of the input image.
2. Step 1: Scan the input ECG image column by column.
3. Step 2: Extract the black pixels (pixels that represent the signal).
4. Step 3: In case of the presence of isolated pixels, create one segment with them.
5. Step 4: Place each segment on the original input image
6. Step 5: Show the boundary of the each segment
7. Step 6: Subtract the segment that represents the ECG signal from the other segment.

Smoothing the ECG signal

ECG signal is smoothed by morphological thicken process because of containing residual effects from the subtraction of the background. Morphological operation “thicken” is used to remove the pixels that exist on the boundary of the signal.

BW2 = bwm or ph (BW, operation) applies a specific morphological operation to the binary image BW. With n = Inf, thicken objects by adding pixels to the exterior of objects until doing so would result in previously unconnected objects being 8-connected.

Conversion from waveform image to 1-D signal

Final step of digitization is axis value identification. The following section discusses axis identification system.

Several different types of data plotting were analyzed for the sake of the generality of the methodology proposed herein. The piece of paper used to register the data may contain a grid, a box, one horizontal line and one vertical line, or no axis, but in each case a specific analysis needs to be performed in order to adequately interpret the value of the signal obtained, compensating offsets, etc. There are many methods on axis identification. The most complex and difficult part of the proposed algorithm is the axis identification stage. But the easiest way to identify the axis value is the calculation based on the height and width of input image. In this work, the x and y axis values are calculated based on the input scanned image. ECG scanned paper speed is 25 mm/sec. Therefore 1 sec=25 mm and calibration mark of 1mV amplitude = 10 mm. If the input ECG image is 100 mm width and 40 mm high, then the time scale on x axis must be 4 sec and the amplitude scale on y axis must be 4 mV. The disadvantage of this method is that the calculation of the axis values depends on input image size.

Firstly load scanned ECG signal image (.jpg, .bmp). ECG paper recordings need to be scanned. Scanning resolution can be 600/300/200 dpi (dots per inch). In this work the image is obtained by scanning with 600 dpi resolution. The image from (Figures 6-21) is 600 dpi resolutions. The algorithm uses the function im read ( ) to import the scanned image of the recording as the JPEG file. (Figure 6) represents a scanned ECG paper recording.

Figure 6 A Scanned ECG Paper.

Background removing

This step is applied to remove the background grid of an ECG. Background grid is usually of lighter shade of color than the actual signal waveform. The aim of this step is to cancel the background grid and any other noises from the image. To do so, after the paper based record has been scanned, it is imported into the MATLAB command window. Now the “im2bw ( )” function is used to cancel the background grid. This function converts the image into grayscale first and then using a threshold the image is converted into binaries. It cancels every pixel above that threshold using a threshold value. This procedure is implemented for the image entirely. (Figure 7) shows the enhancement image that yields a better-quality of an image (Figure 8) shows the output of background removing it has some printed characters also.

Figure 7 Enhanced Imaged.

Figure 8 Background Removed Image.

Characters removing

To remove the characters, region-based segmentation method is used. Firstly the Morphological area opening is used to remove from a binary image all connected components (objects) that have fewer than P pixels. Screenshot result of the morphological (open) method is shown in (Figure 9). In the region-based segmentation step, the background region is divided as segment 1 as shown in the (Figure 11). The signal is divided as segment 2 as shown in the (Figure 12). These two segments are subtracted each other. The segment subtraction result is shown in (Figure 13). In the segment subtraction, ECG signal has the surrounded region. So to get the ECG signal only, background removed image is subtracted from the image after segment subtraction and the output of this step is shown in (Figure 13). This signal is implemented as shown in (Figure 14). In (Figure 15) shows that the ECG signals is smoothed by morphological thicken process because of containing residual effects from the subtraction of the background. Finally, the ECG signal is converted from the waveform image to 1D signal. The x and y axis values are calculated based on the input scanned image size. (Figure 16) shows the 1-D digitization signal.

Figure 9 Morphologically area Opened Image.

Figure 10 Background Segment 1.

Figure 11 Signal Segment 2.

Figure 12 Subtraction of Segment 1 and 2.

Figure 13 ECG Signal.

Figure 14 Digital ECG Signal.

Figure 15 ECG Signal.

Figure 16 1-D Digitization Signal.

Simulation results with the input image without character

Screenshot result of the original scanned ECG image is depicted in (Figure 17). The characters and small object are not included in this figure. Figure 18 shows the output of the background removing. It does not have some printed characters. It can remove all the background grids. Figure 19 shows the smoothed ECG signal and Figure 20 shows the ECG digitization signal. Finally the axis identification is performed in order to adequately interpret the value of the signal obtained.

Figure 17 Original Scanned ECG Paper (Without Characters/Small Object).

Simulation results with input image resolution 300dpi

Scan into images using a commercial A4 scanner with 300 dpi. It can be seen that the digitization signal is discontinued. Figure 21 shows the original scanned ECG image with 300 dpi. Screenshot result in Figure 22 shows the background removing. It can be seen that the background grid are removed clearly. Although the background grid can be removed, the characters or small object can’t be removed with 300dpi. Moreover the ECG waveform image cannot be clear and some wave place has been disappeared. The axis identification can be carried out based on input original scanned ECG image with 300dpi. Figure 23 shows the 1-D digitization signal with 300dpi.

Figure 18 Background Removed Image.

Figure 19 Smoothed ECG Signal.

Figure 20 1-D Digitization Signal.

Figure 21 Scanned ECG Signal (with 300dpi).

Figure 22 Background Removed (300dpi).

Figure 23 1-D Digitization Signal (300dpi).

ECG signal digitization algorithm has been developed. The background removing is the main point of this method. The character and small objects removing methods is slightly complicated but it is also needed to absolutely complete the digitization system. In this work, the input image must be scanned with resolution of 600 dpi (dots per inch). The background grid and small objects such as characters can be removed exactly. If the image is scanned with resolution of 300 dpi, the background can be removed clearly. But the ECG signal can’t be extract correctly. The characters also can’t be removed in 300 dpi case. If the scanned ECG image does not have characters or other objects that are not ECG signal, the character and small objects removing step is not needed. Finally the axis identification is performed in order to adequately interpret the value of the signal obtained. An image scanned from a bio-potential paper chart contains uniform or plain background, waveform, text representing amplitude mV/mm and recording speed mm/sec. As shown in the flowchart of the signal extraction and background elimination algorithm, this algorithm mainly depends on binary morphological image operations that will estimate the background. The background then can be subtractedfrom the original binary image. The morphological operation that will remove the unconnected segment in the binary background free image is conducted. This result image is needed to smooth by using morphological thickness. The waveform now is the only object presented in such image. In this research, the grid is removed from the scanned ECG images by using morphological operation. Region based segmentation can result in isolated pixels that represent printed characters or noise. The presented algorithm can remove any color, grid or black background and character or noise.

None.

Author declares that there is no conflict of interest.

1. Saied Seddighi A, Afaghi V, Zali AR, et al. Developing a Bedside Software for Digitizing Paper Based Medical Data in Intensive Care Settings. Global Journal of Health Science. 2011;3(1):1‒10.
2. Tsai HC. Segmentation Graduate Institute of Communication Engineering. National Taiwan University, Taiwan; 2010.
3. Dallas Price. Consultant Cardiologist, How to Read Electrocardiogram (ECG). St Marys Hospital, UK; 2010.
4. Gomes Silva AR, Oliveira HM, Lins RD. Converting ECG and Other Paper legated Biomedical Maps into Digital Signals. International Workshop on Graphics Recognition. 2007. p. 21‒28.
5. Khurshid K, Siddiqi I, Faure C, et al. Comparison of Niblack Inspired Binarization for Methods Ancient Documents. Laboratoire crip5 _SIP, Unversite Paris Descartes, France; 2009.
6. Yuan Kueh H, Marco E, Sivaramakrishnan S. Image analysis for biology. MBL Physiology Course, USA; 2008.
7. Mitra S, Mitra M, Chaudhuri BB. A Rough Set Based Approach for ECG Classification. Transactions on Rough Sets IX. Lecture Notes in Computer Science, Volume 5390. 2008. p. 157‒186.
8. Chebil J, Al Nabulsi J, Al Maitah M. A Novel Method for Digitizing Standard ECG Papers. Proceedings of the International Conferences on Computer and Communication Engineering. Kuala Lampur, Malaysia; 2008.
9. Satapathy SC, Murthy JVR, Prasada R. A Comparative Analysis of Unsupervised K-Means, PSO and Self-Organizing PSO for Image Clustering. International Conference on Computational Intelligence and Multimedia Applications. USA; 2007.
10. Davis FA. ECG. 2005.
11. Widmaier EP, Raff H, Trang K. Human Physiology. The Mechanism of Body Function. 9th ed. USA: Mc Graw Hill; 2004.
12. Xiang R, Wang R. Range image segmentation based on split-merge clustering. 17th ICPR, USA; 2004. p. 614‒617.
13. Tsair K, Hwang LJ, Lin YH, et al. Computer Analysis of Electrocardiograms from ECG paper Recordings. Proceeding of 23rd annual EMBS international conferences. Istanbul, Turkey; 2001.
14.  Morganroth J, Silber SS. How to obtain and analyze electrocardiograms in clinical trial. ANE4. 1999;4(4):425‒433.
15. Zucker SW. Region Growing: Childhood and Adolescence, Computer Vision. Graphics and image Processing. USA; 1976. p. 382‒389.