Introduction
Point-of-care testing (POCT) is widely used in the management of diabetes by health care professionals in clinical settings, as well as by patients themselves (1). Glucometers, portable glucose analysers, are POCT devices recommended by the American Diabetes Association (ADA) for self-monitoring at home, in the field, or in clinical settings (2). With the wide availability of various glucometers, there are persistent attempts to improve the accuracy and precision of glucometer readings, in order to match laboratory values for blood glucose (3). While there have been dramatic improvements in performance regarding blood-sample volume and analysis time, only modest improvements in accuracy have been reported since 1989 (2). Due to significant variation among glucometer devices, groups such as the International Organization for Standardization (ISO) have developed performance guidelines. System accuracy criteria for verification of analytical performance capability is established in the first edition of International Standards, ISO 15197:2003, which states that the acceptable system accuracy should be ± 0.83 mmol/L at glucose concentrations of < 4.2 mmol/L, and ± 20% at glucose concentrations of ≥ 4.2 mmol/L (4). However, the second edition, ISO 15197:2013, includes stricter criteria for accuracy, stipulating that 95% of measured glucose values shall fall within either ± 0.83 mmol/L of the average measured values of the reference measurement procedure at glucose concentrations of < 5.55 mmol/L, or within ± 15% at glucose concentrations of ≥ 5.55 mmol/L (5). In addition, the Hospital Services Quality Standards (HSQS), published in 2011 by the Turkish Republic Ministry of Health, identifies procedures for calibration and quality-control testing, as well as quality-management requirements for POCT devices in general (6). However, neither the ISO nor the HSQS guidelines provide any detailed information for monitoring glucometers in hospital settings.
Some recent studies on the subject have either evaluated the performance of a recently introduced glucometer or compared the accuracy and precision of one glucometer to another (2, 7-10). Few studies have precisely quantified the accuracy, reliability and bias of glucometers that are to be used in a research laboratory (2). However, to our knowledge, there is no study in the literature providing details of how a laboratory should check these processes and devices in practice. Therefore, the aim of this study was to present a routine quality-management program for managing and monitoring the quality of a large number of glucometers used in the clinical setting of a high-volume hospital.
Materials and methods
Subjects
This was a cross-sectional quality-control study that was performed to monitor the quality of 83 glucometers used at Antalya Education and Research Hospital, a high-volume tertiary referral centre. The study was conducted in the centre’s Clinical Chemistry Laboratory by laboratory technicians under the supervision of a laboratory expert in February 2015. The study was performed in two sessions on the same day; 52 glucometers were collected and tested in the laboratory in the morning, and the remaining 31 were collected and tested in the afternoon, so that patients’ glucose levels could continue to be monitored in the various hospital departments.
Internal quality-control samples, supplied by the glucometer manufacturer, and blood samples from two patients, Patient I and Patient II, were used to determine the accuracy of the glucometers. Three levels of internal quality-control materials were measured twice on each glucometer. Duplicate measurements from each instrument were averaged, and these mean values were used for data analysis. Both of the patients had high glucose levels and were inpatients in the endocrinology unit; each patient had been evaluated and diagnosed with diabetes by an endocrinologist. Their blood glucose was measured with both the glucose assay and the glucometers, for the quality management of the glucometers used throughout the hospital, as indicated in the HSQS. Patient I, a 54-year-old male, had a haemoglobin of 152 g/L and a haematocrit of 0.440 L/L. Patient II was a 42-year-old male with a haemoglobin level of 162 g/L and a haematocrit of 0.482 L/L. Both patients provided informed consent for venipuncture, in accordance with the forms of HSQS. Two separate tubes of whole blood were obtained from each patient in the fasting state, to be used in one session of the study. The samples were collected by venipuncture in K3 EDTA-containing tubes for anticoagulation (Isotherm 2 mL, lot 140720, REF IST412, Weihai Hongyu Medical Devices Co., Shandong, China; 1.8 mg/mL K3EDTA).
One of the whole-blood samples from Patient I was centrifuged (10 min, 900 x g), and the separated plasma was loaded on the analyser (AU5800; Beckman Coulter Inc., Brea, CA, USA). Duplicate results were obtained for each sample. The second tube was transferred to the working area as soon as possible to obtain the glucometer assay results. Laboratory staff performed the assay using the whole blood, and this sample was used for all 52 glucometers during the morning session of the study. Patient II was venipunctured for the second session of the study, and these samples were used for the second group of 31 glucometers, for which the same procedures were followed.
Methods
Test procedures were performed by laboratory personnel who were well-trained in the limitations of the test systems, the manufacturer’s instructions, the safety practices, and the study protocol. Measurement procedures were carried out under controlled temperature (21–24 °C) and humidity (31–57%).
The POCT glucometers (Rheamed RA1103; Rheamed Biotechnology Co., Changhua, Taiwan) use an electrochemical detection technique and dry reagent strip technology, with the glucose oxidase method. The system provides quantitative measurements of glucose concentrations from 1.1–33.3 mmol/L.
Three levels of control solutions specifically designed for use with the device were used to check the performance of each glucometer, the test strips and the testing technique. The control measurements for the devices were determined with three levels of control materials, and the results were calculated with two replicates. Rheamed control solutions for hypo control (range 1.83–3.49 mmol/L; lot MH1901), low control (range 4.44–6.66 mmol/L; lot NC0601), and high control (range 16.03–24.03 mmol/L; lot NL0301) were used. Reference method measurements of glucose in patient plasma were performed with an enzymatic UV test, the hexokinase method (lot 6353, Beckman Coulter Inc., Brea, CA, USA) on a Beckman Coulter AU5800 analyzer. The test is linear within a concentration range of 0.6–45.0 mmol/L for serum, plasma, hemolysate and CSF.
Statistical analysis
The reproducibility and the deviation of the results from the average were calculated with the coefficient of variation [CV% = (standard deviation / mean) × 100]. Bland–Altman percentage plots were used to determine a percentage deviation of the average, independent of the concentration range. The difference plot, namely the Bland–Altman plot, is the recommended approach for determining system accuracy because statistical assumptions are minimal, and the percentage of the data points meeting the system accuracy performance criteria, as well as the estimating bias, are easily calculated (5). Also, a Clarke grid and consensus grid plots were used to evaluate the clinical significance of inaccuracies in the measurements of blood glucose concentrations. The Clarke Error Grid Analysis compares the clinical significance of the bias between blood glucose system results and lab reference results. For the analysis, data points are assigned to five zones (A–E) on the error grid. The results, or data points, falling into zones A and B are defined as clinically acceptable, for which any observed bias from lab results would not lead to treatment decisions that may put a patient at risk. As the bias, or difference, increases (zones C, D and E), there is a greater risk of under-treating or over-treating a patient based on the results. Statistical testing was performed with SPSS version 18.0 (IBM Corporation, New York, US) (11, 12). The paired glucose measurements obtained from the POCT devices and the Beckman Coulter AU5800 were compared by linear regression analysis and presented as Bland–Altman plots (4, 5).
Results
Mean glucose concentrations of the control and patient samples were determined with 83 glucometers and the standard deviations (SDs) and coefficients of variation (CV%) are presented in Tables 1 and 2. Table 1 shows the results of the three control samples for glucose concentrations measured on all 83 glucometers, within the indicated intervals. The mean values for the control samples were as follows: 2.51 mmol/L for the hypo-control samples, 5.12 mmol/L for the low-control samples and 18.11 mmol/L for the high-control samples. Table 2 shows the results of the patient samples for glucose concentrations measured in two sessions.
Table 1
Mean glucose concentrations of control samples on glucometers investigated.
Table 2
Mean glucose concentrations for whole-blood samples on glucometers investigated.
|
Group |
Whole-blood glucose concentration | ||
|---|---|---|---|
|
Mean (mmol/L) |
SD |
CV (%) |
|
| Patient 1 | 11.56 | 0.5 | 4.4 |
|
Patient 2 |
10.52 |
0.62 |
5.98 |
| SD – standard deviation, CV – coefficient of variation. | |||
Figures 1, 2 and 3 present the results of all of the POCT control measurements shown in the Bland–Altman diagrams. The figures show that all results were within the reference ranges for each control sample.
Figure 1
Hypo-control measurements. The Bland–Altman diagram shows that control sample results of all glucometers tested are within the expected range of the hypo-control sample. Dashed lines – expected range of the hypo-control sample. Solid line – mean glucose concentration obtained by testing hypo-control samples.
Figure 2
Low-control measurements. The Bland–Altman diagram shows that control sample results of all glucometers tested are within the expected range of the low-control sample. Dashed lines – expected range of the low-control sample. Solid line – mean glucose concentration obtained by testing low-control samples.
Figure 3
High-control measurements. The Bland–Altman diagram shows that control sample results of all glucometers tested are within the expected range of the high-control sample. Dashed lines – expected range of the high-control sample. Solid line – mean glucose concentration obtained by testing high-control samples.
To observe the quality control results of the reference analyser (Beckman Coulter AU5800), internal quality-control glucose measurements were performed on the same day, with results of 5.77 mmol/L for the low-level control sample (reference range 4.66–6.44 mmol/L) and 13 mmol/L for the high-level control sample (reference range 10.61–16.5 mmol/L). The within-run precision and total precision for the hexokinase method are each less than 3%. External quality-control glucose measurements with the analyzer were within the appropriate range in February 2015. The plasma glucose level of Patient I was determined as 12.05 mmol/L, and of Patient II was 10.33 mmol/L.
The Clarke Error Grid Analysis was used for the accuracy quantification of POCT data against plasma glucose values from the Beckman Coulter AU5800. This analysis partitions the total blood glucose range into zones, based in the effects of glucose variations during diabetes treatment (3). The zones are as follows:
Zone A: no effect on clinical action
Zone B: altered clinical action – little or no effect on clinical outcome
Zone C: altered clinical action – likely to affect clinical outcome
Zone D: altered clinical action – could have significant medical risk
Zone E: altered clinical action – could have dangerous consequences (5).
Figure 4 shows that all data points of the study fall into Zone A.
Figure 4
Clarke error grid analysis for results of patients obtained with each glucometer versus the hexokinase method. Zone A – no effect on clinical action. Zone B – altered clinical action, little or no effect on clinical outcome. Zone C – altered clinical action, likely to affect clinical outcome. Zone D – altered clinical action, could have significant medical risk. Zone E – altered clinical action, could have dangerous consequences.
The Bland–Altman percentage plot is presented in Figure 5 for plasma measurements against POCT values. The figure illustrates the comparison of the ISO 15197:2003 bias limits compared to the ISO 15197:2013 criteria for accuracy, and shows that all of the POCT results were within the bias limits for ISO 15197:2003, but eight were outside the limits for ISO 15197:2013.
Figure 5
Difference plot of measured values of 83 glucometers. This modified Bland–Altman plot gives superimposed tolerance bands for both ISO 15197:2003 and ISO 15197:2013 criteria. Solid lines – accuracy criteria of ISO 15197:2013; dashed lines – accuracy criteria of ISO 15197:2003. The horizontal black dashed line represents the mean bias, while the horizontal grey dashed lines are the limits of agreement (bias ± 1.96 SD).
Discussion
The Bland–Altman diagrams show that the control sample results of 83 glucometer devices used in the hospital were within the expected range of the quality-control samples. To our knowledge, this is the first study reporting a model for comparing the results of a patient sample assayed on all of the glucometers used in a high-volume hospital, by analysing the reference method results of the same blood sample on the same day. ISO 15197:2013 recommends Bland–Altman and Clarke grid statistical methods for the data analysis of glucose measurements using capillary blood and plasma samples (5). The guidelines also state that 99% of all results must be in zones A and B of the consensus error grid (5). All data points of the study complied with the requirements of the Clarke error grid, representing accurate and acceptable glucose results. The Bland–Altman plot shows the absolute differences between the glucometer measurement results and the comparison measurement results. All of the POCT results were within the bias limits for ISO 15197:2003. However, results obtained with eight glucometers were outside the bias limits for ISO 15197:2013. For POCT systems, the accuracy criteria of ISO 15197:2003, which specify requirements for glucose-monitoring systems intended for self-testing by laypersons for the management of diabetes mellitus, are internationally accepted. In the recently published revision ISO 15197:2013, for which mandatory compliance is recommended after a 36-month transition period, the criteria for system accuracy are more stringent (4, 6). The ISO’s most recent standards provide a refinement of the performance requirements defined in the ISO 15197:2003 guidelines (1). Freckmann et al. (1) pointed out that system accuracy was evaluated in under-standardized conditions, following the ISO Standard 15197:2003. They suggested that self-monitoring of blood glucose systems can achieve system accuracy that is comparable with professional-use systems when measurements are performed by trained personnel in a standardized and controlled setting. Consequently, it was decided that the glucometers (N = 8) with results outside the limits in this study were out of order, and they were replaced with new glucometers that complied with accuracy criteria.
Wehmeier et al. (13) suggested that regular monitoring of the analytical performance of POCT remains of utmost importance. This may involve daily internal quality-control procedures (if used for patient samples) with the control material provided by the manufacturer to check the precision, as well as frequent comparisons with the traceable measurement procedure. As part of the ongoing quality-control programme in the central laboratory, monthly comparisons are recommended in order to monitor the reliability of measurements and to harmonize results from POCT with the traceable measurement procedure (13).
Many countries have created their own policies, procedures or standards that control the use of POCT, in particular any essential requirements and quality-management issues (14). HSQS, published in 2011 by the Turkish Republic Ministry of Health, states that POCT devices should be used near or at the site where care is delivered to patients. These standards define the scope of POCT quality control, and consistent supervision is required by a biochemistry expert. In line with these standards, routine operational procedures are implemented in our hospital for managing the quality control of glucometers. According to these procedures, each clinic should check the devices daily using the control samples provided by the manufacturer, and the results should be documented. However, the HSQS does not define when and how the laboratory expert should monitor the devices and manage the quality control of all glucometers in a high-volume hospital. Therefore, in this study, a scheme to be followed for the monthly routine management of glucometers has been developed by a laboratory expert. Glucometers used in all of the hospital’s clinics should be delivered to the biochemistry laboratory every month, so that internal quality-control samples (at least two levels) provided by the manufacturer may be measured and recorded. In the interim, the glucose concentration of a patient sample determined by a clinical chemistry analyzer using the hexokinase method should be used for comparison with the results obtained by all the devices. The results should then be documented, as defined in ISO 15197:2013. The POCT biochemistry specialist should use the comparison methods and evaluate the results, and the glucometer should be replaced with an appropriate, pre-tested device if the measurements are not adequate.
All quality-control and comparison assay results obtained in this study were appropriate for evaluating the glucometers used in a hospital with a high-volume clinical setting. The method used in this study can be suggested as a model for laboratory managers, especially in similar high-volume hospitals.
This study had some limitations. One of the most difficult issues in gathering all of the glucometers in the laboratory was the management of patient safety. The laboratory could not ask for all the glucometers of certain departments at the same time, especially in units where patients’ glucose levels should be monitored continuously, such as the intensive care unit and endocrine services, or emergency services, where a new patient could arrive at any moment. Therefore, the laboratory crew had to collect the glucometers and analyse the results in two sessions. Another challenge in our study was comparing the results of glucometer measurements with the reference method results. ISO 15197:2013 recommends comparing capillary blood samples with plasma samples. However, it would be impossible to obtain 83 capillary blood samples all at once. A comparison study done with the same sample would be more effective than using different patient samples. Consequently, we decided to use whole blood with EDTA to obtain both the glucometer measurement results and the reference method results, instead of capillary whole-blood samples, due to the similarity and closeness.
In addition, it should be mentioned that besides routine monthly checks, the glucometers in our hospital are brought to the biochemistry lab for analysis if any inaccuracies are suspected, or if a device has been dropped or an accident has occurred. The control and method comparison procedures are then applied to that device alone, and based on the obtained results, the manager decides whether the device will be removed from service. These procedures might explain why all of the glucometers tested in this study seemed to be so unproblematic.
In conclusion, this study proposes a convenient model for the routine management and monitoring of the quality of a large number of glucometers used in a high-volume clinical hospital setting. The study provides details of how a laboratory, under the leadership of a biochemistry expert, should check the accuracy and reliability of glucometer measurements performed in professional medical centres. We believe global standardization is necessary with regard to this issue.
