Clinical data systems, part 2: components and techniques

Clinical data systems, part 2: components and

Computerisation In part I of this series I offered criteria for an ideal patient record. In this paper, I describe the principles of

computerising clinical data. A clinical data system is a computer-based system that captures, stores, or communicates clinical data to enhance medical decision-making. This definition excludes any system used in a healthcare setting solely for management purposes, even if it stores patient data such as diagnoses or procedure lists. Increasingly the developers of such systems, realising that diagnoses and procedures will be inaccurate unless they are recorded by clinicians and used for clinical purposes, derive such data from a "personbased"’ or clinical system.

Structuring of clinical data Computers cannot understand a natural language such as English: with its richness and variability, there are simply too many ways to say the same thing. In addition, paper medical records include abbreviations, alterations, and as the arrangement of entries on the of handwriting. For entry into a computer, data are best recorded with a "controlled vocabulary" or clinical coding system (see later), each element of the record being split into discrete data items, with the relation between them explicit. Thus, while I simply wrote: "BP: 145/90" in Mrs Smith’s notes last Tuesday, a computer would require a much more precise (and verbose) representation, something like:

contextual clues such

page

or

styles

Patient name: Observer: Place of encounter: Date of encounter: Encounter number: Encounter number: Observation type: Observation value: Observation units:

Encounter number: Observation type: Observation value: Observation units:

Mrs Smith Dr Wyatt

ICRF Clinic, Churchill Hospital, 23/8/94

Oxford

6479201 6479201

Systolic blood pressure (lying) 145

techniques

Systems past and present In the late 1930s Schreiber and Nielson3devised a punched-card system for classifying craniocerebral injuries, but nearly twenty years were to pass before clinical data were computerised-and then only for statistical analysis.4 During the 1960s and 1970s scattered applications emerged,5 and some of the largest systems now running have been in continuous development for over a decade. With the advent of the microcomputer in the 1980s computers became commonplace in all healthcare settings, though most of the clinical systems are to be found in primary care.6 Figure 1 shows the annual number of English-language articles with medicor health and comput- in their title or abstract indexed on Medline, 1966-92 (data 1966-83 from ref 7). Clinical data systems come in many shapes and sizes. Examples in order of increasing sophistication are: systems for reporting procedures (eg, for ultrasound scans8); departmental age/sex/diagnosis register systems;9 clinical audit systems;10 ambulatory clinic systems using paper "encounter" forms;" clinical workstations allowing prescribing, ordering of laboratory tests, and retrieval of results;12 and clinical workstations giving access to a full electronic medical record.’3

Components and techniques opposite of human beings: their fails, their logic and patience are inexhaustible, and they excel at repetitive tasks. However complex the "hardware", computers can do nothing until they are activated by "software" programs that tell them where to obtain information, what to do with it, and where to send it. The "hardware" of a typical clinical data system (figure 2) includes: Data input devices-typically keyboards, though pointing devices such as the mouse are now widespread Computers

are

concentration

the

never

mm Hg 6479201 Diastolic blood pressure 90

(lying)

mm Hg

Of course, no clinician would use a system that required data entry in this fashion, so the computer itself has to convert what clinicians record into a form that can be stored and processed. This structuring of clinical data is difficult-indeed, it is one of the central concerns of the emerging discipline of medical informatics.2

renuu Ut

Biomedical Informatics Unit, Imperial Cancer Research Fund, PO Box 123, Lincoln’s Inn Fields, London WC2A 3PX, UK

(Dr J C Wyatt oM)

publication

Figure 1: Annual number of English-language publications medical computing 1966-92.

on

1609

6%).’9 "Encounter forms" are structured forms that help in both the recording and the transcription process; as noted in article 1, they allow more rapid and accurate data retrieval. "Turnaround documents"16 are encounter forms individually printed by the computer with a patient’s current data, which can be corrected or added to when one sees the patient, and are later transcribed. Characters that are carefully written, preferably in capitals, can be scanned and automatically recognised by computer, but "optical character recognition" (OCR) has

high

error rates on

handwriting;

on

typed material,

systems

Figure 2: Components of a typical computerised clinical system

data

Data output

devices-typically a video display screen ("VDU") printer &bul et; Central processors-the logic and calculating engine of &bul et;

or

the computer &bul et; A temporary

for programs and data, consisting of fast, expensive memory; since its contents are erased if power is lost, this is used only for temporary storage • A permanent store for programs and data-usually one or more magnetic disks, either low-capacity removable "floppy" disks or fixed, high-capacity, fast "hard" disks. Optical data storage on CD-ROM (compact disk readonly memory), WORM (write once, read many) disks, or magnetic tape give slower access to large amounts of data. When data must be portable, as in patient-held records, they may be stored on a small card as an optical stripe, magnetic stripe, or silicon chip ("smart card"). No matter what medium data are stored on, they may become corrupted and unreadable, so a backup store is always necessary Communications through networks to other computers. "Local area networks" usually link computers in the same building; when more distant links are needed, "modem" devices allow use of telephone lines, but "wide area networks" allow faster links and a greater choice of services, such as electronic mail and file transfer through InterNet.14 Clinical data systems occasionally include other components-eg, a yellow flashing light on terminals to alert those nearby to a message, 15 a pill counter linked to the computer to dispense prescriptions and record therapy, 12 or physiological transducers to monitor vital signs. For more details, consult refs 16 and 17. store

Data entry methods: the "user interface" Anyone implementing a clinical data system will be worried about the time clinicians have to spend entering data. To set this in perspective we should not forget, however, that conventional methods are very slow: Lurie et al’8 reported that interns on-call spent a mean of 28 minutes seeing each new patient but 33 minutes recording their observations in the notes-a total of 3 hours per night documenting findings. The time taken to enter data into a computer depends on the methods used; there are six ways to transfer data from a person to a computer

(figure 3). Writing on paper is the most familiar way of recording data, but transcribing this into a computer is labourintensive, time-consuming, and error-prone (error rate 1610

error

per letter are 2-4%. Direct handwriting onto a sensitive computer screen is used in pen-based systems, and can be as accurate as OCR on typed material because it captures the order in which parts of letters are written, but it is only just emerging into clinical use in a$lm pilot project by the US Department of Defense. What about marking? If all possibilities for a data item can be enumerated (eg, sex, marital status), forms can be printed with check boxes beside each option and a scanner can detect which boxes are marked; clinicians will have encountered such "optical mark reading" forms in multiple-choice exams, and they are widely used for anaesthetic audit. 20 Disadvantages are that the special machines require space and regular cleaning. One commercial system, DataFax, captures research data from forms that are scanned and transmitted by fax; at the receiving end, a bar code identifies the form, then the computer locates data fields and boxes and performs optical character recognition on handwriting and optical mark recognition on check boxes. Yet another marking method makes use of a paper overlay on a graphics pad, which captures data by detecting where the mark is made and allows the form to be filed in the clinical notes. Many clinicians resent having to type in their data: although an experienced operator can enter 200 characters a minute with a 4% error rate,2’ most clinicians will reach only 80 (with more errors). Keyboards are also bulky and noisy, and some say hazardous to health.22 Chord keyboards, in which several keys are pressed simultaneously, are more compact and can be faster-but they require training.23 One disadvantage of both is that they encourage entry of free text, rather than entry or rates

Figure

3: Methods for

entering data into

a

computer

selection of defined data items from a list. One system for interviewing patients about dyspepsia uses a specially adapted keyboard to allow patients to answer questions

displayed on the screen.24 Pointing is an intuitive activity, and studies of children at play led researchers to develop the popular Windows, Icons, Menus, and Pointing device ("WIMP") method for communicating with computers. Windows are areas on a high-resolution screen which contain icons-small drawings of objects-that can be selected by a pointing device such as a mouse and manipulated by commands on the pull-down menus.25 This method requires more powerful computers but reduces the need for training, encourages exploration, makes the system more forgiving of user error, and offers explicit support and feedbackvalued qualities in a complex apparatus.26 Other pointing devices include cursor keys on keyboards, trackballs (which require less space than a mouse), touch screens (which are currently expensive low-resolution devices), light pens to point at the screen (users tire of using these faster than with a mouse), and bar-code readers. In a study of 20 nurses entering single-character data, no errors were made with light pens but the mean time taken was 53% longer than with the keyboard;27 the advantage of pointing devices over keyboards for data entry emerge only when data items are longer than one character and can be selected from a pre-enumerated list. Data entry by bar-code is used in blood transfusion laboratories; the staff scan codes from test tubes or from a preprinted sheet of bar-codes, as at the supermarket. One clinical data system, DIOGENE in Geneva, has used speech entry of data since the 1970s,28 but this is indirect: the clinician telephones a transcriptionist who types data that then appear on a terminal near the clinician’s phone and are edited under physician guidance. In the past five years, automatic understanding of speech has become sufficiently affordable and accurate to be used clinically in settings with a limited vocabulary-such as radiology and pathology reporting and accident and emergency29-as long as users are willing to utter their reports one word at a time, pausing for a fraction of a second between words. An accuracy of 97% on a limited vocabulary of 1000-2000 words is attainable, after "training" to an individual’s voice.3° Finally, clinical data such as vital signs may be captured from a patient by recording electrical signals direct, through transducers such as electronic thermometers,

oximeters,

or

pulse

rate meters or

through

an

imaging

device. This can be extremely beneficial in saving nursing time, for example on an intensive care unit, but the methods used to reject artifacts and the resolution of the conversion process are crucial issues (ref 31, for example, for evaluation of digitised chest radiographs).

What should be in the database? A database stores data in a form that computers can read, as in the blood-pressure example. Databases are usually built with commercial programs that help maintain data integrity, provide security mechanisms and data-entry forms, and facilitate rapid data retrieval. Here are some possible criteria for selecting database programs for a clinical data system: Allows import and export of data for interchange with statistics and other packages Use of the industry-standard Structured Query

Language (SQL)

various computers, such as personal Macintosh or UNIX workstations and computers * Supports construction of time-oriented databases * Allows data to be rearranged or viewed in different ways; usually, this means a "relational" database Supports a "distributed data model", with many discrete databases held on linked computers appearing functionally as one large database &bul et; Will support the required number of concurrent users with sub-second response times. Although a standard off-the-shelf database is a wise choice for developers of small systems, several of the larger hospital-wide information systems use custom database software to ensure rapid response times. Some databases allow "client" programs running on workstations to use them as a "server", helping to make the system more user-friendly. If the database can handle images and sounds as well as textual data this may be useful; few yet have this multimedia capability. One major concern is how much storage space is required and how long data should be kept instantly accessible. One 450-bed hospital decided to keep data about laboratory tests and drugs on-line for six months and summaries of patient encounters for fifteen months; this required an on-line storage capacity of 12 gigabytes (12 000 million characters)." It is valuable if there is one central place in each system, the "data dictionary", where the characteristics of all data items are stored. This allows the definition of a data item to be changed and new data items to be added. Such characteristics of data items, or "metadata", include the name of the data item with a description and definition, units in which the item is measured, the classes of user who are authorised to edit or view the item, and whether a full audit trail is to be held for the item. A basic principle of clinical data systems is that clinicians should be free, within limits, to collect the data items they use in decision-making, as long as they do not indulge in excessive "stamp collecting". However, to ensure that the needs of other users and organisations are met, as well as to allow exchange of data, users normally agree to collect a core or minimum dataset. Since collecting and storing data on computers has a cost, clinicians should scrutinise each item to be included in the minimum or core dataset for relevance, repeatability, and ease of acquisition. If leaving some data items on paper records will cause no problems, they should not be computerised; many clinical data systems profitably coexist with paper records. To facilitate data exchange and pooling, definitions of data items in the core dataset should be standardised regionally or nationally. If data are worth recording on a database they will presumably be needed later for decision-making so must be kept safe.32 The main threats to data security are threefold. The first is deliberate tampering through unauthorised access or computer viruses: this can be discouraged by access controls, maintenance of a full "audit trail" of all values a data item takes, and antivirus precautions. The second is inadvertent temporary loss of access through power failure (solution, uninterruptable power supply) or failure of the computer (solution, duplicate entire system or install a "fault tolerant" computer). The third is inadvertent permanent loss of data through data corruption (solution, back-up data onto separate storage, and remove to a fireproof safe or another *

Runs

on

site). 1611

*Eg, clinical findings, diagnoses, drugs, procedures, Table: Characteristics of some clinical

sites of

disease, aetiological factors, occupations.

coding systems

Ensuring confidentiality Clinicians have an ethical and legal obligation to prevent patient data falling into the hands of unauthorised persons-whether by accident or by design. There are various techniques for avoiding breaches of confidentiality. One can restrict physical access to rooms containing terminals and printers. Hardware access controls include magnetic identity cards that authorised users swipe through a slot before logging-in to the system. Software access controls include providing users with secret log-in names and passwords; passwords are never shared or written down and are changed regularly; even when logged-in, users can only access data they need and sensitive data may be retrieved only from certain terminals;12 and some records, or sections of records, can be designated "monitored notes", and those who wish to read them are warned that their identity will be sent to the record author.12 Encryption for data in storage or in transit can be done with coding algorithms whose output can be read only by those with a key. Finally, discarded printouts can be shredded. All these procedures carry an overhead, most noticeable when staff move from terminal to terminal and must login every time. Sensible judgment will indicate which precautions are justified to reduce the risk of unauthorised access to below that of a paper record system.

Ensuring data quality Shared, single entry of data To share out data entry, data items should be entered by the person or system most appropriate. If it concerns a patient’s feelings or habits, ideally the patient should enter it, perhaps using a specially adapted computer terminal; this can be considerably more accurate33 and complete than when a clinician takes the history. Each item of data should be entered once only, which often means linking information systems. This is helped by the move towards "open systems" for databases and communications, but hindered if different identifiers are used on each system. Integration with other systems As with uranium, the more data available in one place, the greater the value. Once clinicians have ready access to a critical mass of data they will use the system. 12 Such data typically include demographics, laboratory results, medications, and discharge diagnoses or problem lists. Links to managerial and financial systems may allow more accurate accounting and complete reimbursement of costs, with consequent savings that pay for the clinical data system. 12 However, there is a potential disadvantage to such integration: if clinicians fear that information on patient outcomes or their own productivity can be extracted by managers, unfairly disregarding case-mix, they may be reluctant to use the system. Equally, they have an ethical duty to ensure that clinical data about identifiable patients 1612

revealed to non-clinical colleagues. To avoid possible defensiveness in data collection, both provider and patient identities can be hidden from certain users on linked systems. Coding and clinical terms Computers lack common sense and cannot understand English: they assume, for example, that "Angina", "Diagnosis: Angina", and "angina" all represent different concepts. If the diagnosis is recorded in free text, anyone who wants to identify all patients with a diagnosis of angina will need to interrogate the database exhaustively with all words and phrases that may have been used to express the concept, including variations due to mis-spellings and capital letters. To a overcome this retrieval problem, "controlled is used: one term is used vocabulary" only internally for each concept-it may be a recognisable English word or an alphanumeric code-and a "mapping table" links synonyms and abbreviations to the core term. The actual synonym used may be recorded elsewhere on the database to retain fidelity with the clinician’s statement but, internally, the core term takes precedence. Many-too many-clinical coding systems have been developed; some of the best known are listed in table 1. To match clinical requirements, coding systems should cover all the kinds of data found in medical records, should be able to code data in as much or little detail as necessary, and should be logically structured, consistent, unambiguous, widely used, and able to incorporate new medical concepts (eg, new drugs) without difficulty. Many of the current systems were designed to classify diverse patients into a small number of classes for epidemiological or reimbursement purposes, so are inadequate for clinical tasks; guidelines for selecting clinical coding systems have recently been assembled by the American Medical Informatics Association. Since data are best entered by those closest to their source, coding by clinicians should lead to more accurate data than coding by trained clerks from written records, partly because it allows clinicians to express concepts at the right level of detail. For example, if the coding system is hierarchical, when a clinician selects a diagnosis of "asthma", subtypes such as "occupational asthma" will be offered which may be selected instead. Clinicians will feel confident selecting the more precise diagnosis, while coding clerks will not. Prevention and control of error As already mentioned, data entry by keyboard is associated with a 4% error rate.2’ If the error concerns a single blood pressure, this may not be serious; if it is a radiotherapy dosage, it may. The size of errors varies considerably: in cancer registry data’ dates of birth and death were 93% accurate with mean errors of one day, but the radiotherapy dose for patients with bladder carcinoma was accurate in only 84%, with a mean error of 14 Gray-one-third of the normal dose. Surprisingly, error rates are not always is

not

reduced as transcribers learn; there was no change over 20 weeks in one study.2’ Error rates did increase as the "response set"-the number of possible choicesincreased. Although error rates varied from 2 to 10% and transcription rates from 30 to 130 records per hour between transcribers, transcribers with the lowest error rates were also the fastest. Finding the right people to transcribe may be one solution to error control, but other steps may also be taken:42 prevention, by careful form design and clear definition of each data item; detection of illegal entries by validity checks within each data item (absolute range checks, a "verify" range outside which the system asks for confirmation, a check on legal data types or a dictionary of legal entries); correction of incorrect but legal values by means of redundancy/consistency checks across data items (eg, sex & diagnosis, dates of birth & appointment), use of check digits (eg, on identifiers), "delta" check (reject data if rate of change is greater than some plausible value); detection of incorrect (but legal) values by double-keying the whole dataset (error rate falls as the square of the original error rate); and compensation (eg, standardise on upper case for names and map words onto those that sound the same with the Soundex algorithm). As with maintaining confidentiality of data, it is necessary to balance the stringency of error checks against reductions in speed and usability of the system.21 Authentication It is a simple task to arrange that every entry in a clinical database is attributable and timed, since users must log-on before being granted access and computers can automatically time-stamp data. However, there are occasions-such as the issue of a prescription by computer or the discharge of a patient-when extra evidence of the identity of the user may be required. This "electronic signature" may be a code number from a list of numbers used once only and in sequence, or a second password. Such precautions, together with access controls, ensure that the authenticity of orders is at least as certain as that of a hand-written note; even so, they are not yet accepted by all jurisdictions as a substitute for a

signature.

preventive care: provide reminders about preventive activities, automatic calculation of prognostic scores Better

Better communication: allow staff to communicate with others by electronic mail (9000 messages were sent per week in one hospital); send automatic message if patients are admitted under another doctor12 Access to other information resources: Medline and other national databases,43 local phone directories, &c. Ability to search through stored data to discover or explore clinical hypotheses44 Better administration: easier calculation of audit

figures;

more

complete billing.

Displaying clinical data A sheet of paper contains four times as much data as one screen of a text-based terminal, and printed reports are read 25% faster than the same text displayed on a VDU screen.45 As people become more familiar with computers and modern screens these differences may narrow, but paper still has the advantages that it can be annotated and is portable. Whether generating paper or on-screen reports, computers have a useful function in searching for relevant data and arranging them persuasively. Many graphical methods have been used for displaying clinical data, including Russell’s Condition Diagram,46 and various charts (eg, the fetal partogram). The design of charts does matter: a randomised evaluation of different formats for partograms showed that chart format made differences clear to decisions reached by 16 obstetricians.47 Powsner and Tufte48 lately proposed that we should summarise the mass of data in a patient’s clinical record, including findings, laboratory results, and therapies, by use of multiple small graphs: this is an attractive notion but it has not been evaluated. One potential hazard is that non-linear time-scales and widely differing scaling for the Y axis might disorientate clinicians looking for periodic relationships or comparing laboratory results and drug dosages.

Conclusion clinicians be rewarded for using

How

can

new

system?

a

Clinicians are busy people who spend much of their time with patients, move around, are constantly interrupted by the telephone or a bleep, discuss patients with other team members, and are rightly intolerant of externally imposed restrictions. They need a clinical record system that delivers immediate advantages over paper records to compensate for the extra difficulty of data entry. Such incentives might be: Better quality documentation: problem-focused flow sheets and summaries; pre-written paragraphs for clinic letters Faster ordering of laboratory tests: forms pre-filled according to the clinical context * Integration of data from multiple sources * Better information from laboratories: faster access to results results; flagged if abnormal or greater than 20% change from a previous result; results presented as a table or graphical chart. Better quality prescribing: allow doctors to order drugs on-line, provide automatic dosage calculation, reminders about drug interactions and allergies

It is up to individual clinicians to judge what features are needed in their system, to determine the correct balance between ease of data retrieval and security, and to say whether the huge costs of a hospital-wide system with hundreds of terminals are warranted. As a profession too, we must decide what kinds of clinical data systems we need, before "computer programmers cast institutional convenience into silicon". 48 The final article will discuss how to develop and evaluate a clinical data system.

References

Leaning M. The new information management and technology strategy of the NHS. BMJ 1993; 307: 217. 2 Greenes RA, Shortliffe EH. Medical informatics: an emerging academic discipline and institutional priority. JAMA 1990; 263: 1

1114-20. Schreiber F, Neilson A. A punch card code for the classification of craniocerebral injuries. J Michigan Med Soc 1938; 37: 909-12. 4 Hollingsworth TH. Using an electronic computer in a problem of medical diagnosis. J R Stat Soc A 1959; 122: 221-31. 5 Sittig DF, Stead WW. Computer-based physician order entry: the state of the art. J Am Med Informatics Assoc 1994; 1: 108-23. 6 Barnett GO, Jenders RA, Chueh HC. The computer-based clinical record: where do we stand? Ann Intern Med 1993; 119: 1046-48. 7 Collen MF. Origins of medical informatics. West J Med 1986; 145: 3

778-85.

1613

Kuhn K, Gaus W, Wechsler JG, Janowitz P, et al. Structured reporting of medical findings: evaluation of a system in gastroenterology. Meth Inf Med 1992; 31: 268-74. 9 Jones RB, Hedley AJ. A computer in the diabetic clinic: completeness of data in a clinical information system. Practical Diabetes 1986; 3: 295-96. 10 Barrie JL, Marsh DR. Quality of data in the Manchester orthopaedic database. BMJ 1992; 304: 159-62. 11 McDonald CJ, Hui SL, Smith DM, et al. Reminders to physicians from an introspective computer medical record. A two-year randomized trial. Ann Intern Med 1984; 100: 130-38. 12 Bleich HL, Beckley RF, Horowitz GL, et al. Clinical computing in a teaching hospital. N Engl J Med 1985; 312: 756-64. 13 Rind DM, Safran C. Real and imagined barriers to an electronic medical record. In: Safran C, ed. Proceedings of 17th Annual Symposium of Computer Applications of Medical Care. Washington: AMIA, 1994: 74-78. 14 Glowniak JV, Bushway MK. Computer networks as a medical resonance: assessing and using the Internet. JAMA 1994; 271: 1934-39. 15 Bradshaw KE, Gardner RM, Pryor TA. Development of a computerised laboratory alerting system. Computers Biomed Res 1989; 22: 575-87. 16 Shortliffe E, Perrault L, Wiederhold G, Fagan L, eds. Medical informatics. Wokingham: Addison Wesley, 1990. 17 Blum BI. Clinical information systems. New York: Springer Verlag, 1986. 18 Lurie N, Rank B, Pareenti C, Wooley T, Snoke W How do house officers spend their nights? N Engl J Med 1989; 320: 1673-77. 19 Wilton R, Pennisi AJ. Evaluating the accuracy of transcribed clinical data. In: Safran C, ed. Proceedings of 17th Annual Symposium on Computer Applications in Medical Care. Washington: AMIA, 1994: 279-83. 20 Emberton M, Meredith P. Data capture: caught in the act. Br J Healthcare Comp 1993; 10: 32-34. 21 Norton SL, Buchanan AV, Rossman DL, Chakraborty R, Weiss KM. Data entry errors in an on-line operation. Computers Biomed Res 1981; 14: 179-98. 22 Brahams D. Repetitive strain injury. Lancet 1993; 342: 1168. 23 Oliver RTD, Leahy M, Wyatt J. Evaluation of five-key keyboard based palm-top machine for medical history and clinical trial data collection. In: Forum on clinical computing. London: Royal Society of Medicine, 8

1993. 24 Lucas

RW, Card WI, Knill-Jones RP, Watkinson G, Crean GP.

Computer interrogation of patients. BMJ 1976; ii: 623-25. 25 Johnson J, Roberts TL, Verplank W, et al. The Xerox Star: a retrospective. Computer 1989 (September): 11-27. 26 Norman D. The design of everyday things. New York: Doubleday, 1990. 27 Murchie

CJ, Kenney GNC. Comparison of keyboard, light pen and recognition as methods of data input. Int J Clin Monitoring Comput 1988; 5: 243-46.

video

1614

28 Sherrer JR, Baud RH, Hochstrasser D, Osman R. An integrated hospital information system in Geneva. MD Computing 1990; 7: 81-89. 29 Holbrook J, Aghababian R. A computerised audit of 15,009 emergency department records. Ann Emerg Med 1990; 19: 139-44. 30 Shiffman S, Wu AW, Poon AD, et al. Building a speech interface to a medical diagnostic system. IEEE Expert 1991 (February): 41-50. 31 Lams PM, Cocklin ML. Spatial resolution requirements for digital chest radiographs: an ROC study of observer performance in selected cases. Radiology 1986; 158: 11-19. 32 Smith MF. Are clinical information systems safe? BMJ 1994; 308: 612. 33 Locke SE, Kowaloff HB, Hoff RG, et al. Computer-based interview or screening blood donors for risk of HIV transmission. JAMA 1992; 268: 1301-05. 34 Chisholm J. The Read clinical classification. BMJ 1990; 300: 1092. 35 Buckland R. The language of health. BMJ 1993; 306: 287-88. 36 Rothwell DJ, Cote RA, Cordeau JP, Boisvert MA. Developing a standard data structure for medical language: the SNOMED proposal. In: Safran C, ed. Proceedings of 17th Annual Symposium on Computer Applications of Medical Care. Washington: AMIA, 1994: 695-99. 37 Lindberg DA, Humphreys BL, McCray AT. The unified medical language system. Meth Inf Med 1993; 32: 281-91. 38 US Health Care Financing Administration. The international classification of diseases, 9th revision, clinical modification. Vols 1-3. DHHS publication no PHS80-1260. Washington DC: US Government Printing Office, 1980. 39 American Medical Association. Physicians’ current procedural terminology, 4th ed. London: OPCS, 1986. 40 Office of Population, Censuses and Surveys. Classification of surgical operations, 4th ed. London: OPCS, 1986. 41 Gulliford MC, Bell J, Bourne HM, Petruckevitch A. The reliability of cancer registry records. Br J Cancer 1993; 67: 819-21. 42 Schwartz RJ, Weiss KM, Buchanan AV. Error control in medical data. MD Computing 1985; 2(2): 19-25. 43 Shortliffe EH. Medical informatics and clinical decision-making: the science and the pragmatics. Med Decis Making 1991; 11 (suppl): S2-S14. 44 Safran C, Porter D, Lightfoot J, et al. ClinQuery: a system for online searching of data in a teaching hospital. Ann Intern Med 1989; 111: 751-56. 45 Gould J, Grischkowsky N. Doing the same work with hard copy and with cathode ray tube computer terminals. Human Factors 1984; 26: 323-38. 46 Russell IJ, Hendricson WD, Harris GD, Gobert DV A comparison of two methods for facilitating clinical data integration by medical students. Acad Med 1990; 65: 333-40. 47 Cartmill RSV, Thornton JG. Effect of presentation of partogram information on obstetric decison-making. Lancet 1992; 339: 1520-22. 48 Powsner SM, Tufte ER. Graphical summary of patient status. Lancet 1994; 344: 386-89.