CURRENT ISSUES
Accuracy of Reporting Endocervical Component Adequacy—A Continuous Quality Improvement Project Janie Roberson, S.C.T. (ASCP), Kathy Connolly, C.T. (ASCP), Kay St. John, S.C.T. (ASCP), Isam Eltoum, M.D., and David C. Chhieng,
Inaccurate reporting of the absence of an endocervical (EC) component on Pap smears often results in slide rescreens, amended reports, clinician dissatisfaction, and sometimes unnecessary repeat smears. Therefore, the accuracy of reporting EC component adequacy was selected as a quality indicator for the laboratory continuous quality improvement program (CQI). The process consisted of problem identification, analysis of the situation, collection of data, implementation of solutions, and evaluation of results. The objective of the study was to determine if the accuracy of reporting EC component adequacy on Pap smears improved after application of such a program. During the first phase, 150 Pap smears originally reported with the absence of an adequate EC component and 150 smears reported with the presence of an adequate EC component were rescreened to measure the baseline accuracy of EC component adequacy reporting. The improvement process was then implemented. A cause-and-effect diagram was developed and root cause was determined. A presentation was then made to the cytology staff. Criteria for EC component adequacy were reviewed, examples were shown, and standardized marking of EC component was implemented. Following improvement actions, a second audit of 150 Pap smears reported with the absence of an adequate EC component as well as 150 smears reported with the presence of an adequate EC component was undertaken to measure change in performance in assessing EC component adequacy. For the baseline rescreening, before initiation of the CQI program, 98% accuracy was achieved with smears that were reported as adequate for EC component present. However, the accuracy with smears reported as absence of an adequate EC component was only 71%, i.e., an adequate EC component was identified in almost 1/3 of these cases on rescreen. After the implementation of improvement actions, the accuracy with smears reported with the presence of EC component re-
Department of Pathology, University of Alabama at Birmingham, Birmingham, Alabama Presented in part at the 90th Annual Meeting of the American Society of Cytopathology, Kansas City, MO, November, 6th to 9th, 2001. *Correspondence to: David Chhieng, MD, Assistant Professor, University of Alabama at Birmingham, Department of Pathology, 619 19 St S, KB 627, Birmingham, AL 35249-6823. E-mail:
[email protected] Received 25 January 2002; Accepted 17 May 2002 DOI 10.1002/dc.10162 Published online in Wiley InterScience (www.interscience.wiley.com). © 2002
WILEY-LISS, INC.
M.D.*
mained high (98%) and the accuracy of reporting the absence of EC component was 90%. The difference of the latter before and after the implementation was statistically significant (P ⫽ 0.015, z-test). The accuracy of reporting EC component adequacy increased following the CQI process. Using reporting EC component adequacy as an example, we demonstrate that by treating clinical problems as quality control issues and applying basic quality improvement tools, a positive outcome can be effected. Diagn. Cytopathol. 2002;27:181–184. ©
2002 Wiley-Liss, Inc.
Key Words: cervicovaginal smears; endocervical component; adequacy; continuous quality improvement
Gynecologic cytology screening programs have rendered cervical cancer a preventable disease. Over 90% of invasive cervical carcinoma can be avoided by periodic examination of cervicovaginal (Pap) smears. For cervical cancer screening to be effective, all components of this complex, multidisciplinary process must function well. Hence, monitoring of every step of the process is important. There is increasing focus on the outcomes and processes. Such activities aimed at monitoring and improving the process and outcome are often termed continuous quality improvement (CQI) or total quality management (TQM). The steps of CQI, as outlined by the Joint Commission on the Accreditation of Hospital Organization (JCAHO), include, 1) identification and evaluation of the quality indicators by well defined data collection; 2) assessment of data collected to determine areas of improvement; 3) planning and implementation of improvement action; 4) evaluation of the impact of improvement action; and 5) redesign of the processes to build upon areas of improvement.1 In gynecologic cytology, the process includes all the steps that occur from specimen collection to delivery of the final report to the health care providers and their patients. In this report, we describe our experience of applying the CQI process to a project aimed at improving gynecologic Diagnostic Cytopathology, Vol 27, No 3
181
ROBERSON ET AL.
cytology service in our laboratory. The accuracy of reporting endocervical (EC) component on Pap smears was selected as the quality indicator to be evaluated. This particular quality indicator was chosen because there were increasing requests from clinicians to review smears that were reported as lacking an adequate EC component and a significant number of these smears were found to have adequate numbers of EC cells on review. After initial analysis and collection of data, improvement actions were implemented. We then evaluated the impact of such improvement actions on the accuracy of reporting EC component adequacy.
Materials and Methods The project adopted the framework as outlined by JCAHO.1 To better understand and identify the sources of variation within the process, a cause-and-effect diagram was developed. The latter identified the factors (causes) that lead to the outcome (effect) and was expressed as a fishbone diagram. The factors were activities which resulted in the outcome being analyzed. Related factors were grouped under the same category or major “bone” in the fishbone diagram. In our project, there were five major “bones”— People, Materials, Machines, Methods, and Environment. The next step was to identify the most likely root cause— the one that occurred frequently and/or appeared to have a significant impact on the outcome. We verified our choice of root cause by collecting and analyzing the data. Data collection consisted of retrieving a set of 150 consecutive Pap smears originally reported as without an adequate EC component and another set of 150 smears reported as satisfactory with an adequate EC component from the archives of the Cytopathology Laboratory at the University of Alabama at Birmingham during a 1-mo period. All smears were conventional and were collected using an endocervical brush, an Ayre spatula, or a cervical brush, depending on the preferences of the healthcare providers who obtained the smears. An adequate EC component was defined as the presence of a minimum of 10 EC or squamous metaplastic cells in groups or singly according to the Bethesda System 1991.2 These 300 smears were rescreened to assess the adequacy and accuracy of reporting of EC component. The results represented the baseline accuracy of EC component adequacy reporting in our laboratory. After measuring the problem and determining the root cause, improvement actions were devised and implemented. Three months following the implementation of improvement actions, a second audit was performed and consisted of rescreening two sets of 150 conventional Pap smears; one set was originally reported with an absence of an adequate EC component and the other set as satisfactory with the presence of an adequate EC component during a 1-mo period. The result was then compared with that of the first 182
Diagnostic Cytopathology, Vol 27, No 3
Fig. 1. Cause-and-effect diagram.
audit to assess the impact of the improvement action. Data was analyzed using the z-test. Level of significance was set at less than or equal to 0.05.
Results The process for improvement in this project was related to the accuracy of reporting EC cells adequacy in Pap smears. The accuracy of reporting EC cells adequacy was the quality indicator. Once the indicator was identified, a cause-andeffect diagram was developed (Fig. 1). For example, under the category of “method,” possible causes included lack of a clear understanding of EC component criteria by cytotechnologists. Under the category of “people,” one of the causes was failure of the healthcare providers to provide accurate and/or relevant clinical information, such as specimen source, cervical vs. vaginal, and prior history of hysterectomy. Another “people” cause was incorrect reporting of the EC component by the cytotechnologists. After “brain storming,” we identified that inaccurate identification and reporting of the EC component by cytotechnologists was the most likely root cause in the current situation. During the first audit, before the implementation of any improvement action, 98% accuracy of reporting the presence of an adequate EC component on Pap smears was achieved with smears that were originally reported as adequate for EC component present. However, the accuracy with smears originally reported with absence of an adequate EC component was only 71%, i.e., an adequate number of EC cells were identified in almost one-third of these cases on rescreening (Fig. C-1). An educational presentation was then made to the cytotechnologists and included an explanation of the improvement project, the results of the first audit, and the improvement actions. The latter consisted of a review of the criteria of an adequate EC component and examples of EC components that were overlooked (Fig. 2) as well as implementation of standardized marking of EC cells. Three months following improvement actions, the accuracy of reporting the absence of an adequate EC component on Pap smears with smears that were originally reported as absence of an adequate EC component was 90%. When
ENDOCERVICAL COMPONENT ADEQUACY
Fig. C-1. Examples of endocervical cells that were missed on initial screening.
Fig. 2. Accuracy of reporting EC component before and after the implementation of improvement actions.
compared to the first audit, the improvement was statistically significant (z test, p ⫽ 0.015).
Discussion There is an increasing emphasis on quality improvement in healthcare resulting from the application of management concepts and practices developed outside of the healthcare industry to the practice of medicine. Following World War
II, W. Edward Deming and others developed the ideas of total quality management (TQM).3 Managers and workers were encouraged to collaborate in the evaluation of the actual steps in the processes of production, rather than an end-product inspection. They also advocated that the examination of the production process should be a continuous operation to improve productivity and product quality— hence, the term “continuous quality improvement.” It is agreed that CQI was the major reason for Japan’s postwar economic success. The basic principles of CQI consist of 1) focus on the process or activity, rather than the individual worker or department; 2) emphasis on customer satisfaction and feedback; 3) continuous and systematic evaluation and improvement processes leading to improved product quality, reduced waste, and lowered costs.4 The concept of CQI was largely ignored by the healthcare industry until the mid1980s. As defined by Kristchevsky and Simmons,5 a system is the result of a series of activities and interactions between individuals that result in the production of a product or delivery of a service. It is not surprising that there has been growing interest and pressure in the application of CQI to the healthcare industry. With increasing pressures to reduce costs while improving quality of the service, the National Demonstration Project on Quality Improvement in Health Care was formed in 1987 to address specific quality problems that had been resistant to solutions such as billing errors.6 Other major national organizations such as the JCAHO and International Academy of Cytology have also adopted or advocated the use of CQI approaches as part of quality assurance programs.1,7 Laboratory service is one of the many hospital-based activities that can benefit from CQI. We conducted this project to evaluate and resolve inconsistency related to the reporting of EC component in Pap smears in our laboratory. Although the presence of an adequate EC component is considered by the Bethesda system as an indicator of specimen quality, its relationship with the detection of precancerous lesions in Pap smears has been a matter of debate. The majority of cross-sectional studies have demonstrated that the rate of detection of epithelial abnormalities is significantly lower in smears without an adequate EC component than in smears with EC cells.8 –12 However, longitudinal studies do not support this observation.13–16 The authors of these studies observed that patients with smears lacking an adequate EC component initially did not have a higher rate of histologic high-grade abnormality when compared to patients who had satisfactory Pap smears. The ACOG interim guidelines in 1994 stated that the absence of an adequate EC component alone should not be the only criterion for repeating Pap smears.17 Nonetheless, reporting the absence of an adequate EC component often impacts patient management, sometimes by initiating early repeat smears. There is no consensus on Diagnostic Cytopathology, Vol 27, No 3
183
ROBERSON ET AL.
whether the practice of repeat Pap smear is justified. To reflect the current position on the management of patients with an inadequate EC component and/or other obscuring factors, Bethesda 2001 eliminated the category “Satisfactory but limited by,” instead recommending classifying smears with an inadequate EC component and/or other obscuring factors as “Satisfactory” with a quality indicator statement.18 The most common reason for reporting an inadequate EC component is a lack of transformation elements secondary to sampling difficulties. However, in our situation failure to recognize or report the presence of EC component was also a significant problem. This was reflected by the fact that there were increasing requests from clinicians to review smears that were reported as lacking EC smears and a significant proportion of these smears were found to have adequate numbers of EC cells on review. This resulted in slide rescreens, amended reports, clinician dissatisfaction, and sometimes unnecessary repeat smears. All were translated to a substantial increase in the consumption of resources and time. Furthermore, there is significant inconvenience and emotional toll on patients when a repeat Pap smear is performed. Simply instructing cytotechnologists to be more vigilant in identifying and correctly reporting EC component might have produced the same results and cost far less time and effort. However, without conducting the CQI project we would not be able to quantitatively document the result. Furthermore, if there was no improvement in the accuracy of reporting EC component, we would have been less likely to have a full understanding of the processes involved and, in turn, less likely to develop an alternative strategy to resolve the issue if no improvement was observed. The use of the fishbone diagram allowed us to focus on the entire process in detail and provide a better understanding of the factors that affect the outcome. It also aided us in implementing improvement actions as well as data collection and analyses after the potential root cause was identified. Data was not available for the rate of smears that did not have an adequate EC component before and after the implementation of the CQI program. As a matter of fact, any changes in the incidence of smears without an adequate EC component may not necessarily reflect the change in the accuracy of reporting EC cells because other factors such as skills of the healthcare providers and characteristics of the patient population can also play an important role. In conclusion, we reported our experience in utilizing CQI to resolve a laboratory quality issue. By adopting an
184
Diagnostic Cytopathology, Vol 27, No 3
approach that was data-driven and process-oriented, we were able to measure, correct, and improve the technical process and the quality of the services our laboratory offered.
References 1. Organization JCotAoH. Using quality improvement tools in a healthcare setting. Oakbrook Terrace, IL: JCAHO, 1992. 2. Kurman RJ, Solomon D. The Bethesda system: for reporting cervical/ vaginal cytologic diagnoses. New York: Springer-Verlag;1994. 3. Deming WE. The new economics for industry, government, education and health care. Cambridge, MA: Massachusetts Institute of Technology, Center for Advancing Engineering Study; 1993. 4. Van Matre JG. The D*A*T approach to total quality management. J Ahima 1992;63:38 – 40. 5. Kritchevsky SB, Simmons BP. Continuous quality improvement. Concepts and applications for physician care. JAMA 1991;266:1817–1823. 6. Berwick DM, Godfrey AB, Roessner J. Curing health care: new strategies for quality improvement. A report on the National Demonstration Project on Quality Improvement in Health Care. San Francisco: JosseyBass Health Series; 1990. 7. Mody DR, Davey DD, Branca M, et al. Quality assurance and risk reduction guidelines. Acta Cytol 2000;44:496 –507. 8. Elias A, Linthorst G, Bekker B, Vooijs PG. The significance of endocervical cells in the diagnosis of cervical epithelial changes. Acta Cytol 1983;27:225–229. 9. Gilbert FE, Hicklin MD, Inhorn SL, et al. Letter: standards of adequacy of cytologic examination of the female genital tract. Am J Obstet Gynecol 1973;117:863– 864. 10. Mitchell H, Medley G. Cellular differences between true negative and false negative Papanicolaou smears. Cytopathology 1993;4:285–290. 11. van der Graaf Y, Vooijs GP, Gaillard HL, Go DM. Screening errors in cervical cytologic screening. Acta Cytol 1987;31:434 – 438. 12. Vooijs PG, Elias A, van der Graaf Y, Veling S. Relationship between the diagnosis of epithelial abnormalities and the composition of cervical smears. Acta Cytol 1985;29:323–328. 13. Mitchell HS. Longitudinal analysis of histologic high-grade disease after negative cervical cytology according to endocervical status. Cancer 2001;93:237–240. 14. Mitchell H, Medley G. Longitudinal study of women with negative cervical smears according to endocervical status. Lancet 1991;337: 265–267. 15. Bos AB, van Ballegooijen M, Elske van den Akker-van Marle M, Hanselaar AG, van Oortmarssen GJ, Habbema JD. Endocervical status is not predictive of the incidence of cervical cancer in the years after negative smears. Am J Clin Pathol 2001;115:851– 855. 16. Mintzer M, Curtis P, Resnick JC, Morrell D. The effect of the quality of Papanicolaou smears on the detection of cytologic abnormalities. Cancer 1999;87:113–117. 17. Anonymous. ACOG committee opinion. Absence of endocervical cells on a Pap test. Number 153, March 1995. Committee on Gynecologic Practice. American College of Obstetricians and Gynecologists. Int J Gynaecol Obstet 1995;49:212. 18. Solomon D, Davey D, Kurman R, et al. The 2001 Bethesda system: terminology for reporting results of cervical cytology. JAMA 2002; 287:2114 –2119.
