CLINFO — A Successful Technology Transfer

CLINFO

-

A SUCCESSFUL TECHNOLOGY TRANSFER

Susan F. Whitehead and Martin Streeter Bolt Beranek and Newman Inc. BBN Software Products One Alewife Place, Cambridge, MA 02140

tor in the preparation and analysis of data, physical inaccessibility or inconvenience of "central" computer facilities, and the investigator's loss of control over the data once they were turned over to a central processing facility.

ABSTRACT CLINFO, a data management and analysis system for clinical research, was originally developed under NIH sponsorship and then made commercially available. After five years of deployment and support by BBN, the system is being productively utilized by hundreds of investigators in 40 major medical institutions. An ongoing development program has resulted in CLINFO-PLUS, a major reimplementation of the software that brings extensive enhancements to the system. CLINFO is a shining example of the successful transfer of governmentsponsored technology into the real world.

These survey findings led the NIH to contract with Rand to develop a prototype data management and analysis system designed to address these needs. The resulting CLINFO prototype was fieldtested and extensively evaluated at three clinical research centers. The system proved so successful at these three sites that the NIH proceeded with plans for wider dissemination by sponsoring a competition for the development of CLINFO as a commercially viable system. Bolt Beranek and Newman Inc. (BBN) won that competition and began deploying CLINFO systems in 1979. The results of the first year of commercial deployment were reported at the SCAMC meeting in 1980[2].

Introduction The CLINFO system was developed under the sponsorship of the National Institutes of Health (NIH) during the mid-1970's to meet the need for an appropriate tool for management and analysis of clinical research data. In its prototype form, CLINFO was extensively evaluated in three clinical research centers and found to be highly useful and widely accepted by investigators. This result led the government to move towards making this new technology more widely available to clinical investigators, and CLINFO became a commercially deployed and supported system in 1979, with Bolt Beranek and Newman Inc. as the purveyor. After five years of dissemination of the CLINFO system, it is timely to convey to the medical/computer community the results of what has proven to be an unusually successful transfer of governmentsponsored technology into the real world.

System Features CLINFO's user-friendly features have been described in detail in previous presentations[2,3J and in the CLINFO Reference Manual[4J. CLINFO is a minicomputer-based information system designed specifically for the clinical research setting. It runs on Digital Equipment Corporation (DEC) VAX hardware with the VMS operating system, as well as on PDP-11's with RSX-11M. CLINFO is designed to operate as an interactive, time-shared system; sites typically have from two to six or more terminals for simultaneous use. Both DEC and Tektronix terminals are currently supported, and the system will soon be terminal-independent.

Backwround

By design, CLINFO is simple to learn and easy to use for clinical investigators and other research personnel with little or no prior computer experience. CLINFO is a prompt-driven system, with menus of options and step-by-step prompting for the information necessary to specify a particular data operation. There is thus no need to remember complex commands or to learn a programming language in order to perform the basic functions of data management and analysis. The system also includes a built-in "HELP" facility, and utilizes programmable keys on some terminals to provide "push-button" access to system functions.

CLINFO grew out of an effort on the part of the Division of Research Resources (DRR) of the NIH to identify the computer needs of clinical investigators using the DRR-supported General Clinical Research Centers. A comprehensive survey conducted by the Rand Corporation in 1973 showed that despite the widespread availability of computer resources in most major medical centers, many investigators did not avail themselves of these resourcestl]. The survey identified a number of clear barriers to convenient use of existing computer facilities, including the level of technical expertise needed to interact with such systems, the lack of technical assistance to the investiga-

CH2090-9/84/0000/0557$01 .00 ©

1984 IEEE

CLINFO is organized functionally into "activities" corresponding closely to the informationrelated activities already familiar to the inves-

557

tigator involved in the clinical research process. The major functions of these activities are outlined in Figure 1, and include the ability to "describe" the organization of one's study data, to "enter" the data into a database, to "retrieve" selected data from the database, and to perform "analyses" and other calculations on the data. Activity

sheets, CLIk4FO data may be subjected to various user-specified calculations and to a set of familiar statistical analyses (t-test, chi-square, regression, analysis of variance, etc.). The data may also be displayed graphically in such forms as scatter plots, connected-line graphs, bar graphs, regression curves with confidence bands, and survival curves (see Figure 2).

Maor Functions

deacribe

nae, characterize, and group patient parameters to be measured and stored in database

enter

enter patient data into database

examine

display and edit data in database

aubset

establiah subgroups of patients in database, based

retrieve

aove selected portions of database into worksheet for display and analysis

worksheet

create worksheets, enter data into them, sort the

on

specified conditions applied

data, calculate

and

to

Worksheet: (Last modified on 24 Jul 84) E systolic blood pressure predicted by age

160 T

0

140

their

data

~~~~~~~

00 _,-ft

__Q--

swat

Y-Aied a...........

0,< 06

110

LoW CI

display them 165

perform arithmetic, logioal, and oumulative on rows or columns of a worksheet

High C:

operations analyze

90 go

perform simple parametric and non-parametric statistical procedures and other oomputations on worksheet data, and display data or results graphically

response files

create, edit, and oeecuto lists of sequential user responses to system prompts, for repetitive data entry and other frequently used sequences of activities

program

files

execute an external progrm and return to CLINFO list worksheets, subsets, and response files

mesaage

record messages for System Manager

Figure 1.

25

a

sheet

from

a

40

45

50

55

60

65

In addition to the basic features

described

above, CLINFO offers users several more sophisticated capabilities, including the ability to automate repetitive operations without the need for programming by utilizing response files (i.e., saved-answer files), and the ability to easily move data between CLINFO and other computer systems (in either direction) via communication files (i.e., system-generated ASCII files). CLINFO's capabilities can be augmented by the use of programming languages (e.g., Basic or Fortran) to write procedures for functions not already part of the CLINFO activities. Further, the inclusion in the system of a screen-oriented text editor and formatter makes word processing capabilities for document preparation easily available own user

to investigators on the their data reside.

computer

same

which

on

Status of Dissemination In the first five years

of

commercial

deploy-

ment, BBN has installed 36 CLINFO systems, in addition to converting the four prototype sites. There are at least another eight new installations planned over the coming several months. The locations of these sites

Data are manipulated by the investigator in two-dimensional, table-like structures called "worksheets". These may be created either by retrieving data from the database, or by typing data in directly, and are analogous to a piece of or

35

Figure 2. CLINFO Regression Curve

CLINFO Activities and their Functions

paper

30

age

Data for a particular research protocol or study are permanently stored in a hierarchical, time-oriented database called the Study Data File. This database typically contains for each patient a series of measurements over time for some set of The investigator describes the attrivariables. butes and organization of data variables for his or her own study via a data dictionary called the "schema". This schema information is then used to screen data during entry, for example, for acceptable codes or for values falling within a specified range. CLINFO's unique understanding of the time element in clinical research enables the investigator to retrieve data which are time-aligned across patients relative to some event of importance in the study, e.g., the administration of a particular drug, or the time of the peak value of some measurement.

scratch

i

w

ure

3; they clearly

are cover

shown

on

the map in

Fig-

all parts of the country

and include many outstanding medical institutions.

Thirty-eight of the sites are at clinical research centers (GCRCs) supported by the NIH, which continues to take an active interest in the development, support, and dissemination of the system.

lab notebook.

Worksheets are temporary structures that may be created and deleted at will, without affecting the permanent oopy of the study data in the carefullyprotected Study Data File. In the form of work-

CLINFO is seen at these sites as adding a new dimension to the centers, and is often regarded as a valuable institutional resource by medical school faculty and administrators. The ten non-GCRC

558

sites include two V.A. medical centers, two pharmaceutical companies, a cardiology department, aa psychiatry department, a biomedical engineering9 department, a nutrition research center, a genetics research group, and a private research hospital.

computer systems extensively. On average, the database capabilities of CLINFO are used by about 60% of the users, while the remainder simply manipulate their data in worksheets.

Usage data from 19

of the GCRC sites for FY83 total of almost 75,000 connect hours on CLINFO, up from about 56,000 for 14 GCRC sites in FY82. These centers supported an average of 41 studies or protocols on CLINFO during the year, and an average of 57 investigators per center. The extensive use of the system is also reflected in the increasing numbers of journal publications

indicate

*Mapa CUf Ses

a

resulting from CLINFO

use. Annual reports from the GCRCs to the NIH this past year contained references to more than a thousand abstracts and papers published from research done using CLINFO.

Figure 3.

CLINFO sites typically have an on-site person called a System Manager, who plays both a facilities management and training role, and a consultative role with respect to advising investigators on the organization and analysis of their data. The CLINFO System Managers organized themselves formally in 1981 as the Organization of CLINFO System Managers (OCSM), and have been meeting as a group annually to discuss such topics as enhancements to existing system features, future directions for growth of the system, and the evolving role of the System Manager. These meetings also provide a valuable opportunity for exchanges between sites regarding approaches to particular types of research problems, and for seminars to enable System Managers to better help users with

Map of CLINFO Sites

The computer hardware at the existing sites includes 24 VAXes and 14 PDP-11's, some of which are in the process of converting to VAXes. As shown in Figure 4, these CLINFO sites have over 175 terminals, each serving multiple users, so that there are now literally hundreds of clinical investigators accessing CLINFO. In addition to clinicians, the users include research fellows, nurses, dieticians, lab technicians, medical students, administrators, and others; the system has been enthusiastically accepted by all of these.

CLINFO statistics.

SYstem DeveloDment

CLINFO has had an ongoing program of system enhancements in response to the user community. In the first several years of deployment, enhance-

ments primarily took the form of refinements of and extensions to existing capabilities, e.g., easing of limits on numbers of patients and variables, step function curves for life table analysis, more flexibility in printing of files and results, and so forth. However, there were also several major new developments, such as a screenoriented schema editor, a forms data entry package with forms automatically generated from the schema design, hierarchical patient retrievals, and ex-

CLINFO Sites and Terminals October 1984 Sites

Terminals

GCRC's

31

141

Non-GCRC's

9

35

Total

40

176

panded terminal support.

In order to better meet the needs of a grow-

Figure 4.

ing user community, BBN last year began a major reimplementation of the software, called CLINFOPLUS. This reimplementation, scheduled for release to existing sites in early 1985, will bring to the system some fundamental improvements to the worksheet and database structures, will take advantage of now computing and display hardware, and will increase the extensibility and maintainability of the code.

CLINFO Sites and Terminals

Profile of User ConmunitY CLINFO users represent a variety of research interests: endocrinology, cardiology, oncology, nephrology, nutrition, immunology, psychiatry, geographic medicine, animal studies, and so forth. A recent poll of the sites showed that about 60% of CLINFO users represent individuals who have had no prior computer experience before being introduced to the system; only 10% have previously used

are:

Some examples of CLINFO-PLUS enhancements text and numerics stored in the same work-

sheet, codelists that oan contain 32,000 code entries per list, derived items created from items 559

are also becoming more instrumental in teaching investigators the rudiments of good experimental design.

anywhere in the database, subsets that are automatically updated with the addition of new data, terminal independence, enhanced graphics capabilities (including support for color graphics), an integrated user programming facility that allows direct access to worksheets and graphs, and userfriendly interfaces to packages such as SAS and The new CLINFO-PLUS software is largely BMDP. written in the "C" language, and in a BBNdeveloped interpretive language known as RPL -the Research Programming Language.

Another factor for success has been the close communication within the CLINFO community. System Managers at the various sites keep in touch with each other not only through the OCSM meetings but also via phone (contacts at each site are maintained in a CLINFO phone directory). Research staff such as dieticians also communicate with each other about shared interests and special procedures developed on CLINFO. System Managers share special-purpose programs, as well as analysis techniques. A CLINFO Newsletter keeps the community apprised of new installations, development efforts, sharable programs, and other system news.

Future directions for CLINFO development will be driven by the user community, and are likely to encompass several specific areas. As clinical research becomes more and more data-oriented and thus requires more statistics, there undoubtably will be some additional statistical capabilities added to the system. For example, some multiple comparison tests have already been included. However, the original design philosophy of limiting CLINFO's statistics to those that most clinical investigators can utilize appropriately has proven sound, and it is unlikely that a great many additional statistics will be added to CLINFO itself. Growth in this area is better addressed by providing direct interfaces to statistical packages such as SAS and BMDP.

Certainly, however, the real success of any system is directly related to the results that users see. The system provides a well-integrated and comprehensive set of tools for data management and analysis in a user-friendly environment that is readily accessible to the investigator. As pointed out in the 1982 SCAMC report from Vanderbilt on five years of CLINFO use[3J, investigators feel they have gained a lot in terms of research insights through use of the system, and are attaining quicker and more publishable results. Many find that due to the earlier insights into trends in their data, they can reduce the elapsed time needed to complete a study and publish the results.

There is an increasing interest at many sites in using CLINFO for clinical trials and other collaborative research, and it is therefore likely that inter-site communications will become an increasingly pressing topic in the user community. There has also been a great deal of interest in the areas of dietary analysis and administrative aids for the GCRCs. Some System Managers have already begun developing such programs, and it is anticipated that there will be more and more sharing of site-developed applications over time.

CLINFO, then, itself a shining example of the successful transfer of government-sponsored technology into the real world, has now become the means of facilitating further technology transfer of research results. The system has lived up to its original concept by the NIH, and has proven to be a viable and widely-accepted tool for clinical investigation.

Reasons for Success The success of the CLINFO system can in large measure be attributed to the ongoing commitment from the NIH and BBN to making the system responsive to the needs of clinical research and the CLINFO user community. The original vision of the NIH in providing a system that would be easily accessible to investigators, easy for them to learn and use, and that would come with appropriate technical assistance has proven to be right on target and has endured as a viable model over more than seven years of use (for the prototype sites).

REFERENCES

[1) Palley, N.A. and Groner,

G.F., "Information Processing Needs and Practices of Clinical Investigators -- Survey Results", AFIPS Conf. Proc. 444:717, 1975.

[2] Whitehead, S.F. and Bilofsky, H., "CLINFO

-

A

Clinical Research Data Management and Analysis System", Proceedings of the Fourth Annual Svmposium on ComDuter ADDlications in Medical Care, Nov 1980.

The CLINFO System Manager role is probably the single most important factor in the success of the system. The ready availability to the investigator of someone who can not only teach him or her how to use the system, but also assist in the design of the database and in how to analyze the data has proven an invaluable part of the CLINFO Over time, the teaching role of the environment. System Manager tends to move away from the teaching of the "nuts and bolts" of system use (which users help each other with), and more towards the teaching of analysis techniques and statistics. The System Managers, as well as the system itself,

t3] Johnston, H.B., Higgins, S.B., Harris, T.R.,

and Lacy, W.W., "Five Years Experience with the CLINFO Data Base Management and Analysis Systemn, Proceedings of the Sixth Annual SrPosium on Computer Annlications in Medical Care, Nov 1982.

[4] CLINFO Reference Manual, Bolt Beranek and Newman Inc., Cambridge, MA, 1980.

560