Outcomes research primer

CLINICAL EPIDEMIOLOGYIOUTCOMES RESEARCH JAY F. PICCIRILLO, MD Clinical Epidemiology/Outcomes Research Associate Editor

Outcomes research primer JAY F. PICCIRILLO, MD, MICHAEL G. STEWART,MD, MPH, RICHARD E. GLIKLICH, MD, and BEVAN YUEH, MD, MPH, St. Louis, Missouri, Houston, Texas, Boston, Massachusetts, and New Haven, Connecticut

O u t c o m e s research is the scientific study of the outcomes o f diverse therapies used for a particular disease, condition, or illness. 1,2 The goals of this type of research are to document treatment effectiveness, to create treatment guidelines, and to study the impact of insurance status or reimbursement polices on outcomes of care. 3,4 Increasingly, otolaryngology providers are being asked by third-party payers, managed care organizations, and other organizations to document treatment outcomes. The American A c a d e m y of O t o l a r y n g o l o g y - H e a d and Neck Surgery ( A A O - H N S ) Foundation has recently begun a research grant program specially targeted to support outcomes research projects in otolaryngology, Each y e a r m o r e instruction courses on outcomes research axe offered at the annual A A O - H N S meeting. Clearly, the A A O - H N S membership are more interested in outcomes research than ever before. For that reason, we decided to prepare this Primer on outcomes research. It is our hope that this primer will provide the interested reader with a "starting" point to explore the published literature on outcomes research. We do not claim that this primer is exhaustive and admit that many excellent articles may have been omitted. Furthermore, we recognize that the field of outcomes research continues to grow, and we plan periodic updates to include new articles describing these new developments.

From the Departments of Otolaryngology-Head and Neck Surgery, Washington University School of Medicine (Dr. Piccirillo), Baylor College of Medicine (Dr. Stewart), Massachusetts Eye and Eat" Infirmary (Dr. Gliklich); and the Section of Otolaryngology, Department of Surgery and the Robert Wood Johnson Clinical Scholars Program, Department of Medicine Yale University School of Medicine (Dr. Yueh). Dr. Yueh is a Robert Wood Johnson Clinical Scholars Fellow. Reprint requests: Jay F. Piccirillo, MD, Box 8115, 517 S. Euclid Ave., Washington University School of Medicine, St. Louis, MO 63110. Otolaryngol Head Neck Surg 1997;117:380-7. Copyright © 1997 by the American Academy of OtolaryngologyHead and Neck Surgery Foundation, Inc. 0194-5998/97/$5.00 + 0 23/1/82130 380

The primer is divided into three main parts: Key Articles, General Articles, and Otolaryngology-specific Articles. As the name implies, the Key Articles section includes those articles we believe are fundamental to the field of outcomes research. These articles are, or will be, classics in the field. Because of their importance, we have provided a short annotated bibliography to help the reader decide whether the article is of particular importance to him or her. The General Articles section contains articles of a general medical nature, and the Otolaryngology-specific section contains articles pertaining to otolaryngology. REFERENCES 1. Piccirillo JF. Outcomes research and otolaryngology. Otolaryngol Head Neck Surg 1994;111:764-9. 2. Roper WL, Winkenwerder W, Hackbarth GM, Krakauer H. Effectiveness in health care. An initiative to evaluate and improve medical practice. N Engl J Med 1988;319:1197-202. 3. Epstein AM. The outcomes movement--will it get us where we want to go? N Engl J Med 1990;323:266-9. 4. Relman AS. Assessment and accountability.The third revolution in medical care. N Engl J Med 1988;319:1220-2. KEY ARTICLES I. Bergner M. Quality of life, health status, and clinical research. Med Care 1988;27:S148-56. The traditional end points of clinical research are morbidity and mortality, and not quality of fife. Clinical information on the relative usefulness and sensitivity of quality-of-life instruments is often lacking. The author discusses these issues and makes suggestions about techniques and investigations to assist with these problems. 2. Cella DF, Bonomi AE. Measuring quality of life: 1995 update. Oncology 1995;9:47-60. This is an excellent review article that discusses the definition of quality of life and statistical and methodologic issues with quality-of-life assessment. The authors provide a summary of a large number of quality-of-life instruments. 3. Charlson ME, Pompei R Ales HL, et al. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 1987;40:373-83. The authors describe the validation of a weighted comorbidity scaling system and its usefulness in predicting 1-

OtolaryngologyHead and Neck Surgery Volume 117 Number 4

year morality in patients with breast cancer. Basically, comorbid conditions (diseases other than the disease of interest) are weighted according to severity, and an index score is calculated that assists in prospectwe prognostic stratification. 4. Chassin MR, Kosecoff J, Park RE, et al. Does inappropriate use explain geographic variations in the use of health care services? JAMA 1987;258:2533-7. The authors study the degree of appropriateness of coronary angiography, upper gastrointestinal endoscopy, and carotid endarterectomy using established protocols in three different geographic areas. Across all three geographic regions, the authors note relatively high levels of inappropriate utilization. In addition, they find no relationship between levels of appropriateness and different levels of utilization. Therefore inappropriate use does not explain geographic variations in health care utilization. 5. D'Agostino RB, Kwan H. Measuring effectiveness: what to expect without a randomized control groap. Med Care 1995;33:AS95-105. Randomized controlled trials are considered the ideal way to evaluate treatment efficacy. However, such trials are not always possible, and even when possible, they a r e often performed with such restrictions that they do not provide the true measures of the effectiveness of the treatment in the "real world" or under climcal conditions of usual practice. This article reviews the use of nonrandomized studies to measure effectiveness when a randomized controlled trial is not possible. 6. Donabedima A. The quality of care: How can it be assessed? JAMA t988;260:1743-8. This is an overview of issues in determining quality of health care. Two components of quality (technical performance and interpersonal skills) are discussed with respect to cost, techniques in sampling and measurement, and sources of data. Distinctions between structure, process, and outcomes data are drawn. 7. Eddy DM. Clinical decision making: from theory to practice. Cost-effectiveness analysis. A conversation with my father. JAMA 1992;267:1669-75. The first of a series of articles that introduces principles of cost-effectiveness, including its rationale, methods, and applications, through a series of imaginary conversations. Questions are asked and answered with examples from clinical practice. Avoids use of technical economic language. 8. Eisenberg JM. Economics. JAMA 1995;273:16'70-1. This article highlights major findings about the managed care industry from 1994 to 1995. Studies suggest that favorable patient selection and decreases in inpatient utilization were partly responsible for health maintenance organization savings. The differences in health outcomes between fee-for-service or managed care plans are small; satisfaction outcomes are mixed. 9. Ellwood PM. Shattuck lecture---outcomes management. A technology of patient experience. N Engl J Med 1988;318:1549-56. This special report discusses the problems faced by the American medical system and suggests that a new technology of patient experience will help patients, payers,

PICCIRILLO et aL 381

and providers make rational medical choices. Outcomes management consists of a common patient-understood language of health outcomes. The author describes several different aspects of outcomes management. 10. Epstein AM. The outcomes movement--Will it get us where we want to go? N Engi J Med 1990;323:266-9. The author considers the forces that have brought about the outcomes movement and the direction it has taken. The goals of the movement and likely impediments to its progress are discussed. 11. Feinstein AR. Clinical biostatistics. XLI. Hard science, soft data, and the challenges of choosing clinical variables in research. Clin Pharmacol Ther 1977;22:485-98. The choice of variables for clinical research should be selected based on their importance to the condition under study. The essence of science is not the use of "hard" variables over "soft" variables but, rather, the selection of variables that are reliable and accurate. Soft variables that describe many of the important aspects of clinical care can be made more scientific through techniques of clinimetrics and then included in clinical studies. 12. Foundation for Health Services Research. Health outcomes research: a primer. Washington (DC): The Foundation; 1993: This primer provides an overview of health services and outcomes research. 13. Gill TM, Feinstein AR. A critical appraisal of the quality of quality-of-life measurements. JAMA 1994;272:619-26. The authors review the published literature on quality of life and identify the lack of a well-accepted and generalty used definition of quality of life. Authors state that because quality of life is a uniquely personal perspective, patient-specific measures should be used. 14. Greenfield S. The state of outcomes research: are we on target? [editorial]. N Engl J Med 1989;320:1142-3. The author argues that large national databases should be constructed to collect information on the outcomes for patients with different conditions. 15. Guyatt GH, Bombardier C, Tugwell PX. Measuring disease-specific quality of life in clinical trials. Canadian Medical Association Journal 1986;134:889-95. The authors discuss the development, validation, and usefulness of disease-specific quality of life measures in clinical research. A detailed description is provided for techniques of item selection, item reduction, item format, and validation. 16. Iezzoni LI. Severity of illness measures. Comments and caveats. Med Care 1990;28:757-61. This editorial presents information on analyzing severity of illness measures and understanding problems and pitfalls of their use. 17. Kassirer JR The quality of care and the quality of measuring it. N Engl J Med 1993;329:1263-5. This essay about the motivations and intentions behind the movements to establish practice guidelines discusses implications of the rigid applications of these standards. 18. Katz JN, Larson MG, Phillips CB, Fossel AH, Liang MH. Comparative measurement sensitivity of short and longer health status instruments. Med Care 1992;30:917-25. In the study patients are given five different health status measures to assess whether short measures of health stares are as sensitive as longer measures. Overall, short

382 PICCIRILLO et al.

health status measures are more responsive then longer, more established measures. 19. Leape LL, Park RE, Solomon DH, Chassin MR, Kosec0ff J, Brook RH. Does inappropriate use explain small-area variations in the use of health care services? JAMA 1990;263:669-72. Using information from Medicare beneficiaries in several adjacent counties in the same state, the investigators assess whether inappropriate utilization accounts for the large differences in utilization in adjacent counties. Although inappropriate utilization is noted, there are no large differences between counties. Therefore inappropriate use does not explain variation in health care utilization between different counties in the same state. 20. Lohr KN. Outcome measurement: concepts and questions. Inquiry 1988;25:37-50. The need for the accurate measurement of patient outcomes is self-evident. The author describes outcomes research and its use in the measurement of quality of care. The author suggests four areas of improvement to link outcomes to the measurement of process of care. 21. Patrick DL, Deyo RA. Generic and disease-specific measures in assessing health status and quality of life. Med Care 1989;27:$217-32. This article discusses the differences between generic and disease-specific outcomes instruments. Generic measures may be more generalizable and therefore useful for policy studies. Disease-specific instruments may be more sensitive to clinical changes. The authors review considerations selecting an appropriate instrument. 22. Phelps CE. The methodologic foundations of studies of the appropriateness of medical care. N Engl J Med 1993;329:12415. Exploration of potential flaws in studies reporting on appropriateness of care. The article considers how such studies may incorrectly "diagnose" appropriate practice patterns as inappropriate (and vice versa) and draws a mathematical analogy to the imperfect sensitivities and specificities of all diagnostic tests. 23. Piccirillo JR Outcomes research and otolaryngology. Otolaryngol Head Neck Surg 1994; 111:764-9. This overview of the historic development of the outcomes research movement explores the ways outcomes research is different from traditional clinical research. The article also provides four methodologic requirements for outcomes research. Examples of otolaryngology conditions that seem particularly suited to study with outcomes methodology are presented. 24. Piccirillo JF, Feinstein AR. Black-box mathematics and medical practice. Arch Otolaryngol Head Neck Surg 1993; 119:147-55. Two articles presented in the otolaryngology literature using outcomes methodology are presented. These articles present results that are at odds with usual clinical practice. The reasons for this discrepancy are discussed, and an argument is made for improvements in the quality of clinical research provided. 25. Pliskin N, Taylor AK. General principles: cost benefit and decision analysis. In: Bunker J, Barnes B, Mosteller F, editors. Costs, risks and benefits of surgery. New York: Oxford University Press; 1977. p. 5-27. Examples of cost benefit and decision analysis are p r o -

OtolaryngologyHead and Neck Surgery October 1997

vided in this textbook chapter. Concepts such as the time value of money and the use of utility functions to assess nontraditional outcomes are introduced. Several concepts are illustrated with clinical scenarios. 26. Relman AS. Assessment and accountability. The third revolution in medical care. N Engl J Med 1988;319:1220-2. The author describes the historic changes in medicine since 1940 and argues that we have entered a third revolution as a result of uncontrolled expansion in health services and unsustainable growth in expenses. This third era of medicine is based on the consensus for the need for assessment accountability in health care. 27. Roper WL, Winkenwerder W, Hackbarth GM, Krakauer H. Effectiveness in health care. An initiative to evaluate and improve medical practice. N Engl J Med 1988;319:1197-202. A detailed discussion of the effectiveness initiative. Several examples from different clinical conditions are provided to illustrate how outcomes research methodology can improve patient care. 28. Rubin HR, Gandek B, Rogers WH, Kosinsji M, McHorney CA, Ware JE Jr. Patients' ratings of outpatient visits in different practice settings. Results from the Medical Outcomes Study. JAMA 1993 ;270: 835-40. A standard and valid questionnaire of patient satisfaction with their outpatient visit is used to assess satisfaction across general medical practices, such as fee for service, group practice, and prepaid HMO. Implications of patient satisfaction information are provided. 29. Sackett DL, Rosenberg WMC, Gray JAM, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ 1996;312:71-2. This essay argues that evidence-based medicine is the conscientious application of the best available evidence to making decisions about patient care. It includes a description of what evidence-based medicine is, discusses its merits, and counters its most common criticisms. 30. Tarlov AR, Ware JE Jr, Greenfield S, Nelson EC, Perrin E, Zubkoff M. The Medical Outcomes Study: an application of methods for monitoring the results of medical care. JAMA 1989;262:925-30. This important paper describes the methods and results of the Medical Outcomes Study. This was a 2-year observational study designed to help explain how specific components of the health care system affect outcomes. 31. Wennberg J, Gittelsohn A. Variations in medical care among small areas. Sci Am 1982;246:120-34. The authors discuss the differences in utilization rates of many surgical procedures in adjacent counties in Vermont. They conclude that differences are largely the result of the different mix of specialists and the procedures they prefer, rather than the health status of patients. 32. Wennberg JE. Outcomes research, cost containment, and the fear of health care rationing. N Engl J Med 1990;323:1202-4. The authors discuss implications of different rates of health care utilization on health care costs and the potential effects of rationing of health care. The use of outcomes research to identify health outcomes of procedures and medical treatments is less important than society addressing important issues of supply, demand and availability of health care, given fixed budgets and limited resources.

OtolaryngologyHead and Neck Surgery PICCIRILLO et al. 383

Volume 117 Number 4

GENERAL ARTICLES 1. Assessing medical technologies. Washington (DC): National Academy Press; 1985. 2. Committee on Methods for Setting Priorities for Guidelines Development, Institute of Medicine. In: Field M J, editor. Setting priorities for clinical practice guidelines. Washington (DC): National Academy Press; 1995. 3. Aaronson NK. Quality of life assessment in clinical trials. Control Clin Trials 1989;10:195S-208S. 4. Aaronson NK. Quality of life research in clinical trials. A need for common rules and language. Oncology 1990;4:59-66. 5. Anderson JR Kaplan RM, Berry CC, Bush JW Rumbaut RG. Interday reliability of function assessment for a health status measure: the Quality of Well-Being Scale, Med Care 1989;27:1076-84. 6. Apgar V. A proposal for a new method of evaluation of the newborn infant. Anesth Analg 1953;32:260-7. 7. Baldwin LM, Inui TS, Stenkamp S. The effect ~f coordinated, multidisciplinary ambulatory care on service use. charges, quality of care and patient satisfaction in the elderly. J Community Health 1993;18:95-108. 8. Bergner M. Measurement of health statu~. Med Care 1985;23:696-704. 9. Bergner M, Bobbitt RA, Carter WB, Gilson BS. The Sickness Impact Profile: development and final revision of a health stares measure. Med Care 1981;19:787-805. 10. Berwick DM. Continuous improvement as an ~deal in health care. N Engl J M e t 1989;320:53-6. 11. Berwick DM, Godfrey AB, Roessner J. Curing health care. San Francisco (CA): Jossey-Bass; 1990. 12. Bjelle A. Functional status assessment. Curt Opin Rheumatol 1991 ;3:280.-5. 13. Borchardt WW. Patient perception of quality in the clinic setting and relevant strategies for improvement. Cc,llege Review I994;Spring:72-97. 14. Brazier JE, Harper R, Jones NMB, et al. Validating the SF-36 health survey questionnaire: new outcome measure for primary care. BMJ 1992;305:160-4. 15. Brook RH, Lohr KN. Monitoring quality of care in the Medicare program. Two proposed systems. JAMA 1987;258: 3138-41. 16. Brook RH, McGlynn EA. Maintaining qualit~ of care. In: Ginzberg E, editor. Health services research. Key to health policy. Cambridge (MA): Harvard University Press; 1991. p. 284314. 17. Brook RH, 'Ware JE, Davies-Avery A, et al. Overview of adult health status measures fielded in Rand's Health Insurance Study. Med Care 1979;17:1-131. 18. Brook RH, Ware JE, Rogers WH, et al. Does free care improve adults' health? Results from a randomized trial. N Engl J Med 1983;309:1426-34. 19. Busschbach JJV, Horikx PE, van den Bosch JMM, De la Riviere AB, de Charro FT. Measuring the quality of life before and after bilateral lung transplantation in patients with c:/stic fibrosis. Chest 1994; 105:911-7. 20. Caine N, Harrison SC, Sharpies LD, Wallwork 3. Prospective study of quality of life before and after coronary bypass grafting. BMJ 1991;302:511-6. 21. Carey RG, Lloyd RC. Measuring quality improvement in health care. A guide to statistical process control applications. New York: Quality Resources; 1995. 22. Cella DE Cherin EA. Quality of life during and after cancer treatment. Compr Ther 1988;14:69-75. 23. Cella DE Tulsky DS. Measuring quality of life today: methodologic aspects. Oncology 1990;4:29-38. 24. Cella DE Tulsky DS, Gray G, et al. The functional assessment of cancer therapy scale: development and validation of the general measure. J Clin Oncol 1993;11:570-9. 25. Chassin MR, Kosecoff J, Park RE, et al. Indications for selected

26.

27. 28. 29.

30.

31.

32. 33.

34. 35.

36.

37. 38. 39.

40. 41. 42. 43, 44. 45.

46.

47.

48. 49. 50.

medical and surgical procedures: a literature review and ratings of appropriateness: coronary angiography, publication R3204/1-CWF-HF-HCFA-PMT-RWJ. Santa Monica (CA): The Rand Corp; 1986. Chassin MR, Koseeoff J, Solomon DH, Brook RH. How coronary angiography is used. Clinical determinants of appropriateness. JAMA 1987;258:2543-7. Cleary PD, McNeil BJ. Patient satisfaction as an indicator of quality of care. Inquiry 1988;25:25-36. Davies AR, Ware JE. Involving consumers in quality of care assessment. Health Aft (Millwood) !988;7:33-48. Daugherty C, Ratain MJ, Grochowski E, et al. Perceptions of cancer patients and their physicians involved in phase I trials. J Clin Oncol 1995~13:1062-72. de Haes JCJM, van Knippenberg FCE. Quality of life of cancer patients: review of the literature. In: Aaronson NK, Beckmann J, editors. The quality of life of cancer patients: New York: Raven Press; 1987. p. 167-82. Deyo RA, Carter WB. Strategies for improving and expanding the application of health status measures in clinical settings. A researcher-developer viewpoint [discussion]. Med Care 1992;30:MS176-86. Deyo RA, Patrick DL. The significance of treatment effects: the clinical perspective. Med Care 1995;33:AS286-91. Donovan JL, Frankel SJ, Eyles JD. Assessing the need for health status measures. J Epidemiol Community Health 1993;47:15862. Eisenberg JM. Clinical economics. A guide to the economic analysis of clinical practice. JAMA 1989;262:2879-86. Elinson J. In discussion: Donabedian A, Spitzer W, Tartov A. Advances in health assessment conference discussion panel. J Chronic Dis 1987;40(Suppl 1):183S-91S. Elmore JG~ Feinstein AR. A bibliography of publications on observer variability (final installment). J Clin Epidemiol 1992;45:567-80. Epstein AM, Hall JA, Tognetti J, et al. Using proxies to evaluate quality of life. Med Care 1989;27:$91-8. Feinstein AR. The pro-therapeutic classification of co-morbidity in chronic disease. J Chron Dis 1970;23:455-69. Feinstein AR. Clinical biostatistics, XVI. The process of prognostic stratification. Part 2. Clin Pharmaco! Ther 1972; t 3:60924. Feinstein AR. Clinical biostatistics. XIV. The purposes of prognostic stratification. Clin Pharmacol Thor 1972; 13:285-97. Feinstein AR. The 'chagrin factor' and qualitative decision analysis. Arch Intern Med 1985;145:1257-9. Feinstein AR. Para-analysis, faute de mieux, and the perils of riding on a data barge. J Clin Epidemiol 1989;42:929-35. Feinstein AR, Horwitz RI. Double standards, scientific methods, and epidemiologic research. N EngI J Med 1982;307:1611-7. Ferrans CE, Powers MJ. Quality of life index: development and psychometric properties. Adv Nurs 1985;8:15-24. Fowler FJ, Cleary PD, Magaziner J, Patrick DL, Benjamin KL. Methodological issues in measuring patient-reported outcomes: the agenda of the work group on outcomes assessment. M e t Care 1994;32(Suppl):JS65-76. Fries JF, Spitz PW. The hierarchy of patient outcome, in: Spilker B, editor. Quality of life assessments in clinical trials. New York: Raven Press; 1990. p. 25-35. Garnick DW, Hendricks AM, Comstock CB. Measuring quality, of care: fundamental information from administrative datasets. Int J Qual Health Care 1994;6:163-77: Geigle R, Jones SB. Outcomes measurement: a report from the front. Inquiry 1990;27:7-13. Glaser EM. Using behaviora! science strategies for defining the state-of-the-art. J AppI Behav Sci 1980; 16:79-92. Goldberg HI, Martin DP, Christensen DB, Neighbor WE, Inui TS, LoGerfo JR The Health of the Public Program at the University of Washington: a new role for academic medical centers. Acad Med 199l;66:499-505.

OtolaryngologyHead and Neck Surgery October 1997

384 PICCIRILLO et al.

51. Goldmann DA. Contemporary challenges for hospital epidemiology. Am J Med 1991;91:8S-15S. 52. Gonnella JS, Hornbrook MC, Louis DZ. Staging of disease. A case-mix measurement. JAMA 1984;251:637-44. 53. Gotay CC, Korn EL, McCabe MS, Moore TD, Cheson BD. Quality-of-life assessment in cancer treatment protocols: research issues in protocol development. J Natl Cancer Inst 1992;84:575-9. 54. Gouveia WA, Bungay KM, Massaro FJ, Ware JE Jr. Paradigm for the management of patient outcomes [see comments]. Am J Hosp Pharm 1991;48:1912-6. 55. Greco PJ, Eisenberg JM. Changing physicians' practices. N Engl J Med 1993;329:1271-4. 56. Greene MG, Adelman RD, Friedmann E, Charon R. Older patient satisfaction with communication during an initial medical encounter. Soc Sci Med 1994;38:1279-88. 57. Greenfield S, Aronow HU, Elashoff RM, Watanbe D. Flaws in mortality data. The hazards of ignoring comorbid disease. JAMA 1988;260:2253-5. 58. Greenfield S, Kaplan S, Ware JE Jr. Expanding patient involvement in care. Effects on patient outcomes. Ann Intern Med 1985;102:520-8. 59. Greenfield S, Nelson EC. Recent developments and future issues in the use of health status assessment measures in clinical settings. Med Care 1992;30:MS23-41. 60. Griner PF, Mayewski RJ, Mushlin AI, Greenland R Selection and interpretation of diagnostic tests and procedures. Ann Intern Meal 1981;94:553-600. 61. Guyatt G, Feeny D, Patrick D. Issues in quality of life measurement in clinical trials. Control Clin Trials 1991;12:81 S-90S. 62. Guyatt GH, Cook DJ. Health status, quality of life, and the individual [commentary]. JAMA 1994;272:630-1. 63. Harrell FE, Lee KL, Califf RM, Pryor DB, Rosati RA. Regression modelling strategies for improved prognostic prediction. Star Med 1984;3:143-52. 64. Hays RD, Anderson R, Revicki D. Psychometric considerations in evaluating health-related quality of life measures. Qual Life Res 1993;2:441-9. 65. Hays RD, Kravitz RL, Mazel RB, et al. The impact of patient adherence on health outcomes for chronic disease patients in the Medical Outcomes Study. J Behav Med 1994;17:347-60. 66. Hays RD, Stewart AL. Construct validity of MOS health measures. In: Stewart AL, Ware JE, editors, Measuring functioning and welt-being: The Medical Outcomes Study approach. Durham (NC): Duke University Press; 1992. p. 325-42. 67. Hays RD, Stewart AL. Sleep measures. In: Stewart AL, Ware JE, editors. Measuring functioning and well-being: the Medical Outcomes Study approach. Durham (NC): Duke University Press; 1992. p. 235-59. 68. Health Services Research Group. A guide to direct measures of patient satisfaction in clinical practice. Can Med Assoc J 1992; 146:1727-31. 69. Horwitz RI, Feinstein AR. The application of therapeutic-trial principles to improve the design of epidemiologic research: a case-control study suggesting that anticoagulants reduce mortality in patients with myocardial infarction. J Chronic Dis 1981;34:575-83. 70. Iezzoni LI. Using risk-adjusted outcomes to assess clinical practice: an overview of issues pertaining to risk adjustment. Ann Thorac Snrg 1994;58:1822-6. 71. Iezzoni LI, Daley J, Heeren T, et al. Using administrative data to screen hospitals for high complication rates. Inquiry 1994;31:40-55. 72. Iezzoni LI, Foley SM, Heeren T, et al. A method for screening the quality of hospital care using administrative data: preliminary validation results. Qual Rev Bull 1992;18:361-71. 73. Iezzoni LI, Greenberg LG. Widespread assessment of riskadjusted outcomes: lessons from local initiatives. Jt Comm J Qual Improv 1994;20:305-16. 74. Iezzoni LI, Restuccia JD, Shwartz M, et al. The utility of sever-

75.

76. 77,

78.

79.

80.

81.

82.

83.

84.

85.

86. 87. 88. 89.

90.

91. 92.

93. 94.

95.

96. 97.

ity of illness information in assessing the quality of hospital care. The role of the clinical trajectory. Med Care 1992;30:428-44. Jaeschke R, Guyatt GH. How to develop and validate a new quality of life instrument. In: Spilker B, editor. Quality of life assessments in clinical trials. New York: Raven Press; 1990. p. 47-57. Jenkins CD. Assessment of outcomes of health intervention. Soc Sci Med 1992;35:367-75. Jenkins CD, Jono RT, Stanton BA, Stroup-Benham CA. The measurement of health-related quality of life: major dimensions identified by factor analysis. Soc Sci Med 1990;31:925-31. Juniper EF, Guyatt GH, King DR. Comparison of methods for selecting items for a disease-specific quality of life questionnaire: importance versus factor analysis. Qual Life Res 1994;3:52-3. Kahn KL, Rogers WH, Rubenstein LV. Measuring quality of care with explicit process criteria before and after implementation of the DRG-based prospective payment system. JAMA 1990;264:1969-73. Kahn KL, Roth CE Kosecoff J, et al. Indications for selected medical and surgical procedures: a literature review and ratings of appropriateness: diagnostic upper gastrointestinal endoscopy, publication R-3204/4-CWJ-HF-HCFA-PMT-RWJ. Santa Monica (CA): Rand Corp; 1986. Kaplan MH, Feinstein AR. The importance of classifying initial co-morbidity in evaluating the outcome of diabetes mellitus. J Chronic Dis 1974;27:387-404. Kaplan RM, Anderson JR Wu AW, et al. The Quality of WellBeing Scale: applications in AIDS, cystic fibrosis, and arthritis. Med Care 1989;27:$27-43. Katz S, editor. The Portugal Conference: measuring quality of life and functional status in clinical and epidemiological research. J Chronic Dis 1987;40:459-650. Khans WA, Draper EA, Wagner DE Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med 1985;13:818-29. Kosecoff J, Kanouse DE, Rogers WH, et al. Effects of the National Institutes of Health Consensus Development Program on physician practice. JAMA 1987;258:2708-t3. Krahn M, Gafni A. Discounting in the economic evaluation of health care interventions. Med Care 1993;31:403-18. Krause NM, Jay GM. What do global self-rated health items measure? Med Care 1994;32:930-42. L'Abbe KA, Detsky AS, O'Rourke K. Meta-analysis in clinical research. Ann Intern Med 1987;107:224-33. Laffel G, Blumenthal D. The case for using industrial quality management science in health care organizations. JAMA 1989;262:2869-73. Langner SR, Hutelmyer C. Patient satisfaction with outpatient human immunodeficiency virus care as delivered by nurse practitioners and physicians. Holistic Nursing Practice 1995; 10:5460. Leebov W, Vergare M, Scott G. Patient satisfaction: a guide to practice enhancement. Oradell (NJ): Medical Economics; 1990. Lewis CC, Pantell RH, Kieckheffer GM. Assessment of children's health status: field test of new approaches. Med Care 1989;27:s54-65. Lewis JR. Patient views on quality care in general practice: literature review. Soc Sci Med 1994;39:655-70. Light RJ, Pillemer DB. Summing up. The science of reviewing research. Cambridge (MA): Harvard University Press; 1984. Maklan CW, Greene R, Cummings MA. Methodological challenges and innovations in patient outcomes research. Med Care 1994;32(Suppl):JS 13-21. McCormick B. Can research change the way MDs practice medicine? Hospitals 1990;64:32-6. Merrick NJ, Fink A, Brook RH, et al. Indications for selected medical and surgical procedures: a literature review and ratings of appropriateness: carotid endarterectomy, publication R-

OtolaryngologyHead and Neck Surgery Volume 117 Number 4

3204/6-CWJ-HF-HCFA-PMT-RWJ. Santa Monica (CA): Rand Corp; 1986. 98. Mills JL. Sounding board: data torturing. N Engl J Med 1993 ;329:1196-9. 99. Mitchell JB, Bubolz T, Paul JE, et al. Using Medicare claims for outcomes research. Med Care 1994;32(Suppl):JS38-51. 100. Moinpour CM, Feigl R Metch B, Hayden KA, Meyskens FL, Crowley J. Quality of life endpoints in cancer clinical trials: review and recommendations. J Natl Cancer Inst 1!)89;81:485-95. 101. Moloney TW, Paul B. The consumer movement takes hold in medical care. Health Aft (Millwood) 1991 ;10:268-79. 102. Moses LE. Measuring effects without randomized trials? Options, problems, challenges. Med Care 1995;33:AS8-14. 103. Mullan F, Jacoby I. The town meeting for technology: the maturation of consensus conferences. JAMA 1985;254:1068-72. 104. New York State Department of Health. Coronary artery bypass surgery in New York State, 1990-1992. Albany (IX'Y): New York State Department of Health; 1993. 105. Nolan TW, Provost LR Understanding variation. Quality Progress 1990;May:70-8. 106. O'Brien B J, Buxton MJ, Ferguson BA. Measuring the effectiveness of heart transplant programs: quality of life data and their relationship to survival analysis. J Chronic Dis 1987;40:137s-53s. 107. O'Brien B J, Buxton MJ, Patterson DL. Relationship between functional status and health-related quality-of-life after myocardial infarction. Med Care 1993;31:950-5. 108. O'Young J, McPeek B. Quality of life variables in surgical trials. J Chronic Dis 1987;40:513-22. 109. Oye RK, Landefeld CS, Jayes R L Outcomes ir SUPPORT. J Clin Epidemiol 1990;43(Suppl):83S-TS. 110. Palmer RH, Strain R, Maurer J, Rothrock J, Thompson M. Quality assurance in eight adult Medicare practices. Med Care 1984;22:632-43. 111. Park RE, Brook RH, Kosecoff J, et al. Explaining variations in hospital death rates: randomness, severity of illness, quality of care. JAMA 1992;264:484-90. 112. Parkerson GR, Broadhead WE, Tse C-KJ. Comparison of the Duke Health Profile and the MOS Short-Form in healthy young adults. Med Care 1991;29:679-83. 113. Parkerson GR, Broadhead WE, Tse C-KJ. The Duke Health Profile. A i7-item measure of health and dysfunction. Med Care 1990;28:1056-72. 114. Parkerson GR, Connis RT, Broadhead WE, Patrick DL, Taylor TR, Tse C-KJ. Disease-specific versus generic measurement of health-related quality of life in insulin-dependent diabetic patients. Med Care 1993;31:629-39. 115. Parkerson GR, Gehlbach SH, Wagner EH, James SA, Clapp NE, Muhlbaier LH. The Duke-UNC Health Profile: an adult health status instrument for primary care. Med Care 198 l;XIX:806-27. 116. Patton MD. Action research mad the process of continual quality improvement in a cancer center. Oncol Nuts Forum 1993;20: 751-5. 117. Pauker SG, Kassirer JR Decision analysis. N Engl J Med 1987;316:250-7. 118. Payer L. Medicine and culture. New York: Penguin Books; 1988. 119. Pen'y S. The NIH Consensus Development Program: a decade later. N Engl J Med 1987;317:485-8. 120. Powe NR, Turner JA, Maklan CW, Ersek M. Alternative methods for a formal literature review and meta-analysis in AHCPR patient outcomes research teams. Med Care 1994;32(Suppl):JS22-37. 121. Power EJ. Identifying health technologies that work. From the Congressional Office of Technology Assessment. JAMA 1995;274:205. 122. Raskin IE, Maklan CW. Medical treatment effectiveness research: a view from inside the Agency for Health Care Policy and Research. Evaluation and the Health Professions 1991;14:16t-86. 123. Richardson J. Cost utility analysis: what should be measured? Soc Sci Med 1994;39:7-21.

PICCIRILLO et ai, 38.5

124. Robbins JA, Bertakis KD, Helms LJ, Azari R, Callahan EJ, Creten DA. The influence of physician practice behaviors on patient satisfaction. Fam Med 1993;25:17-20. 125. Ross CK, Steward CA, Sinacore JM. A comparative study of seven measures of patient satisfaction. Med Care 1995;33:392406. 126. Ruta DA, Garratt AM, Long M, Russell IT, MacDonald LM. A new approach to the measurement of quality of life: the patientgenerated index. Med Care 1994;32:1109-26. 127. Safran DG, Tarlov AR, Rogers WH. Primary care performance in fee-for-service and prepaid health care system: results from the Medical Outcomes Study. JAMA 1994;271:1579-86. 128. Schag CAC, Heinrich RE. Development of a comprehensive quality of life measurement tool: CARES. Oncology 1990;4:135-8. 129. Schipper H. Guidelines and caveats for quality of life measurement in clinical practice and research. Onco!ogy 1990;4:51-7. 130. Schipper H, Clinch J, McMurray A, Levitt M. Measuring the quality of life of cancer patients: the Functional Living Index-cancer: development and validation. J Clin Oncol 1984;2:47283. 131. Scott IU, Schein OD, West S, Bandeen-Roche K, Enger C, Folstein ME. Functional status and quality of life measurement among ophthalmic patients. Arch Ophthalmol t994;112:32935. 132. Simmer TL, Nerenz DR, Rutt WM, Newcomb CS, Benfer DW. A randomized, controlled trial of an attending staff service in general internal medicine. Med Care 1991 ;29:JS31-40. 133. Sox HC, Blatt MA, Higgins MC, Marton KI. Medical decision making. Stoneham (MA): Butterworths; 1989. 134. Spierer M, Sims HW, Micklitsch CN, Lewis BE. Assessment of patient satisfaction as part of a physician performance evaluation: the Fallon Clinic experience. J Ambulatory Care Manage 1994;I7:1-7. 135. Spitzer WO, Dobson AJ, Hail J, et al. Measuring the quality of life of cancer patients. A concise QL-index for use by physicians. J Chronic Dis 1981 ;34:585-97. 136. Sprangers MAC, Cull A, Bjordal K. The European organization for research and treatment of cancer approach to quality of life assessment: guidelines for developing questionnaire modules. Qual Life Res 1993;2:287-95. 137. Stalfelt AM. Quality of life during induction treatment of acute myeloid leukemia: a comparison of three intensive chemotherapy regimens using three instruments for quality of life assessmont. Acta Oncol 1994;33:477-85. 138. Steinberg ER B ergner M, S ommer A, et al. Variations in cataract management: patient and economic outcomes. Health Serv Res 1990;25:727-31. 139. Stewart AL, Hays RD, Ware JE. Methods of constructing heaith measures. In: Stewart AL, Ware JE, editors. Measuring functioning and well-being: The Medical Outcomes Study approach. Durham (NC): Duke University Press; 1992. p. 67-85. 140. Stewart AL, Hays RD, Ware JE. Health perceptions, energy/fatigue, and health distress measures. In: Stewart AL, Ware JE, editors. Measuring functioning and well-being: The Medical Outcomes Study approach. Durham (NC): Duke University Press; 1992. p. 143-72. 141. Street RL, Gold WR, McDowell T. Using health status surveys in medical consultations. Med Care 1994;32:732-44. 142. Tanenbaum SJ. What physicians know, N Engl J Med 1993 ;329:1268 -70. 143. Teasdale G, Jennett B. Assessment of coma and impaired consciousness: a practical scale. Lancet 1974;2:81-4. 144. Tsvat J, Weeks JC, Guadagnoli E, et al. Using health-related quality of life information: clinical encounters, clinical trials, and health policy. J Gen Intern Med 1994;9:576-82. 145. Tunis SR, Hayward RS, Wilson MC. Internists' attitudes about clinical practice guidelines. Ann Intern Med 1994; 120:956-63. 146. Ware JE. Standards for validating health measures: definition and content. J Chronic Dis !987;40:473-80.

OtolaryngologyHead and Neck Surgery October 1997

386 PICCIRILLO et al.

147. Ware JE, Davies AR, Rubin HR. Patients' assessments of their care. In: Office of Technology Assessment, editor. The quality of medical care: information for consumers. Washington (DC): US Congress; 1988. p. 231-47. 148. Ware JE, Hays RD. Methods for measuring patient satisfaction with specific medical encounters. Med Care 1988;26:393-402. 149. Ware JE, Sherbourne CD. The MOS 36 Item Short-Form Health Survey (SF-36): I. Conceptual framework and item selection. Med Care 1992;30:473-83. 150. Ware JE, Snyder MK. Dimensions of patient attitudes regarding doctors and medical care services. Med Care 1975; 13:60915. 151. Ware JE, Snyder MK, Wright WR, Davies AR. Defining and measuring patients' satisfaction with medical care. Evaluation and Program Planning 1983;6:247-63. 152. Ware JE Jr. Conceptualizing and measuring generic health outcomes. Cancer 1991;67:774-9. 153. Weinstein M, Fineberg H. Clinical decision analysis. Philadelphia: WB Sannders; 1980. 154. Wen SW, Hernandez R, Naylor CD. Pitfalls in nonrandomized outcomes studies. The case of incidental appendectomy with open cholecystectomy. JAMA 1995;274:1687-91. 155. Wennberg J, Gittelsohn A. Small area variations in health care delivery. Science 1973;182:1102-7. 156. Wennberg JE, Barnes BA, Zubkoff M. Professional uncertainty and the problem of supplier-induced demand. Soc Sci Med 1982; 16:811-24. 157. Wennberg JE, Bunker JR Barnes B. The need for assessing the outcomes of common medical practices. Annu Rev Public Health 1980; 1:277-95. 158. Wennberg JE, Roos N, Sola L, Schori A, Jaffe R. Use of claims data systems to evaluate health care outcomes. Mortality and reoperation following prostatectomy. JAMA 1987;257:933-6. 159. Wensing M, Grol R, Smits A. Quality judgements by patients on general practice care: a literature analysis. Soc Sci Med 1994; 38:45-53. 160. Wilson SR. Patient and physician behavior models related to asthma care. Med Care 1993;31 :MS49-60. 161. WinNer JD, Kanouse DE, Brodsley L, et al. Popular press coverage of eight National Institutes of Health consensus development topics. JAMA 1986;255:1323-7. 162. Wray NP, Ashton CM, Kuykendall DH, et al. Selecting disease-outcomes pairs for monitoring the quality of hospital care. Med Care 1995;33:75-89. 163. Yerushalmy J. Statistical problems in assessing methods of medical diagnosis, with special reference to x-ray techniques. Public Health Rep 1947;62:1432-49. 164. Zubrod CG, Schneiderman M, Frei E. Appraisal of methods for the study of chemotherapy of cancer in man: comparative therapeutic trial of nitrogen mustards and triethylene thiophosphoramide. J Chronic Dis 1960;11:7-30.

OTO LARYNGOLOGY-SPECIFIC ARTICLES 1. Consensus Development Panel. NIH Consensus Development Conference Consensus Statement on Acoustic Neuroma. Bethesda (MD): N1H; 1991 Dec 11-13. 2. Early identification of hearing impairment in infants and young children. NIH Consensus Statement, Mar 1-3. 1993;11(1):1-24. 3. Baker CA. Factors associated with rehabilitation in head and neck cancer. Cancer Nurs 1992;15:395-400. 4. Benninger MS, King F, Nichols RD. Management guidelines for improvement of otolaryngology referrals from primary care physicians. Otolaryngol Head Neck Surg 1995; 113:446-52. 5. Bjordal K, Kaasa S, Mastekaasa A. Quality of life in patients treated for head and neck cancer: a follow-up study 7-11 years after radiotherapy. Int J Radiat Oncol Biol Phys 1994;28:847-56. 6. Browman GP, Levine MN, Hodson DI, et al. The Head and Neck Radiotherapy Questionnaire: a morbidity/quality-of-life instru-

7.

8. 9.

10.

11.

12. 13.

14. 15. 16.

17.

18.

19. 20.

21.

22.

23.

24.

25.

26.

27. 28. 29. 30.

ment for clinical trials of radiation therapy in locally advanced head and neck cancer. J Clin Oncol 1993;11:863-72. Bums L, Chase D, Goodwin WJ. Treatment of patients with stage IV cancer: Do the ends justify the means? Otolaryngol Head Neck Surg 1987;97:8-14. Dana ST. Sinusitis panel develops reimbursement criteria [out of committee]. AAO-HNS Bull 1993; 12(12): 11. Fielder H, Denholm SW, Lyons RA, et al. Measurement of health status in patients with vertigo. Clin Otolaryngol 1996;21:124-6. Gliklich RE, Hilinski JM. Longitudinal sensitivity of generic and specific health measures in chronic sinusitis. Qual Life Res I995;4:27-32. Gliklich RE, Metson R. A comparison of sinus computed tomography (CT) staging systems for outcomes research. Am J Rhinol 1994;8:291-7. Gliklich RE, Metson R. Techniques for outcomes research in chronic sinusitis. Laryngoscope 1995;105:387-90. Gliklich RE, Metson R. The health impact of chronic sinusitis in patients seeking otolaryngologic care. Otolaryngol Head Neck Surg 1995;113:104-9. Gotay CC, Moore TD. Assessing quality of life in head and neck cancer. Qual Life Res 1992;1:5-17. Grimby A, Rosenthall U. Health-related quality of life and dizziness in old age. Gerontology 1995;41:286-98. Grundfast KM. Management of otitis media and the new Agency for Health Care Policy and Research guideline. Arch Otolaryngol Head Neck Surg 1994;120:797-8. Hannley M. Academy enters outcomes research arena with new protocol: treatment outcomes for obstructive sleep apnea [news & views from the office of research development]. AAO-HNS Bull 1992;11(9):33. Harris JR Anderson JP, Novak R, An outcomes study of cochlear implants in deaf patients: andiologic, economic, and quality-of-life changes. Arch Otolaryngol Head Neck Surg 1995;121:398-404. Hassan SJ, Weymuller EA Jr. Assessment of quality of life in head and neck cancer patients. Head Neck 1993;15:485-96. Hoffman SR, Dersarkissian RM, Buck SH, Stinziano GD, Buck GM. Sinus disease and surgical treatment: a results oriented quality assurance study. Otolaryngol Head Neck Surg 1989;100:573-7. Hoffman SR, Mahoney MC, Chmiel JF, Stinziano GD, Hoffman KN. Symptom relief after 'endoscopic sinus smgery: an outcomes-based study. Ear Nose Throat J 1993;72:413-4, 419-20. Isenberg SF, Rosenfeld RM. Problems and pitfalls in community-based outcomes research. Otolaryngol Head Neck Surg 1997;116:662-5. Jacobson GP, Newman CW. The development of the dizziness handicap inventory. Arch Otolaryngol Head Neck Surg 1990;116:424-7. Johns ME. The medical appropriateness of tympanostomy tubes proposed for children younger than 16 years in the United States ]editorial comment]. Arch Otolaryngol Head Neck Surg 1994;120:801-2. Juniper EF, Guyatt GH. Development and testing of a new measure of health status for clinical trials in rhinoconjunctivitis. Clin Exp Allergy 1991;21:77-83. Kleinman LC, Kosecoff J, Dubois RW, Brook RH. The medical appropriateness of tympanostomy tubes proposed for children younger than 16 years in the United States. JAMA 1994;271:1250-5. Kuk FK, Tyler RS, Russell D, et al. The psychometric properties of a tinnitus handicap questionnaire. Ear Hear 1990; 1t :434-45. List MA, Ritter-Sterr CA, Baker TM. Longitudinal assessment of quality of life in laryngeal cancer patients. Head Neck 1996;18:1-10. Mangham CA. Decision analysis of auditory brainstem responses and rotational vestibular tests in acoustic tumor diagnosis. Otolaryngol Head Neck Surg 1987;96:22-9. McNeil B J, Weichselbaum R, Pauker SG. Speech and survival.

OtolaryngologyHead and Neck Surgery Volume 117 Number 4

31. 32. 33. 34.

35. 36. 37.

38.

39. 40. 41.

Tradeoffs between quality and quantity of life in laryngeal cancer. N Engl J Med 198l;305:982-7. Meister S. Emerging risk: failure to detect hearing disability in newborns. The problem: lack of screening programs for normal newborns. QRC Advisor 1993;10:1-4. Mulrow CD, Aguilar C, Endicott JE. Quality-of-life changes and hearing impairment: a randomized trial. Ann Intern Med 1990;113:188-94. Mulrow CD, Aguilar C, Endicott JE, et al. Association between hearing impairment and the quality of life of elderly individuals. J Am Geriatr Soc 1990;38:45-50. O'Rourke IC, McNeil RJ, Walker PJ, Bull CA. Objective evaluation of the quality of palliation in patients with oesophageal cancer comparing surgery, radiotherapy and intabation. Aust N Z J Surg 1992;62:922-30. Piccirillo JR Decision analysis in head and neck cancer. Arch Otolaryngol Head Neck Surg 1991; 117:933. Piccirillo JR The inclusion of comorbidity in a staging system for head and neck cancer. Oncology 1995;9:831-6. Piccirillo JF. The use of patient satisfaction data to assess the impact of continuous quality improvement efforts in an academic otolaryngology office. Arch Otolaryngol Head Neck Surg 1996; 122: t 045-8. Piccirillo JE, Edwards D, Haiduk A, Yonan C, Thawley SE. Psychometric and clinimetric validity of the 31-item rhinosinusitis outcome measure (RSOM-31). Am J Rhinol 1995;9:297306. Plante DA, Piccirillo JF, Sofferman RA. Decision analysis of treatment options in pyriform sinus carcinoma. Med Decis Making 1987;7:74-83. Rosenfeld RM. Pilot study of outcomes in pediatric rhinosinusitis. Arch Otolaryngol Head Neck Surg 1995; 121::729-36. Rosenfeld RM, Mandel EM, Bluestone CD. Systemic steroids for otitis media with effusion in children. Arch Otolaryngol

PiCCIRILLO et aL 387

Head Neck Surg 1991;I17:984-9. 42. Rubin HR. Can patients evaluate the quality of hospital care? Med Care Rev 1990;47:267-326. 43. Rubin HR, Wu AW. Patient satisfaction: its importance and how to measure it. In: Gitnick G, editor. The business of medicine: a physician's guide. New York: Elsevier Science Publishing; 1991. p. 397-409. 44. Sher AE, Schechtman KB, Piccirillo JR The efficacy of surgical treatments for obstructive sleep apnea. Sleep 1996; 19:156-77. 45. Snyderman CH, D'Amico E Outcome of carotid artery-resection for neoplastic disease: a meta-analysis. Am J Otolaryngol 1992;13:373-80. 46. The Otitis Media Guideline Panel. Clinical practice guideline, otitis media with effusion in young children. 1994 July. Publication No.: 94-0622. Rockville (MD): Agency for Health Care Policy and Research; 1994. p. iv. 47. Turner RG, Nielsen DV~qApplication of clinical decision analysis to audiological tests. Ear Hear 1984;5:125-33. 48. US Department of Health and Human Services. Managing otitis media with effusion in young children. Arch Otolaryngol Head Neck Surg 1994;120:793-6. 49. Weiss MH, Harrison LB, Isaacs RS. Use of decision analysis in planning a management strategy for the stage NO neck [review]. Arch Otolaryngol Head Neck Surg 1994; 120:699-702.

W e t h a n k Drs. E d w i n M. M o n s e l l , C o o r d i n a t o r o f Research, and Maureen Hannley, Associate Vice President for Research and Development, of the American Academy of Otolaryngology-Head and Neck S u r g e r y F o u n d a t i o n , Inc., f o r t h e i r s u p p o r t o f this work.

Craniofacial and Skull Base Surgery The 2nd Congress on Craniofacial and Skull Base Surgery will he held Jan. 20-25, 1998, in Cancun Mexico. CME credit is available. For further information contact Peggy M. Podboy, CMP, FOCUS Meetings and Events, 625 S George St.,, Mount Prospect, IL 60056-3915; phone (847)398-5821; fax (847)398-5822