Selecting a text-processing system as a qualitative multiple criteria problem

European Journal of Operational Research 50 (1991) 179-187 North-Holland

179

Theory and Methodology

Selecting a text-processing system as a qualitative multiple criteria problem * Leena Tanner Helsinki School of Economics, Runeberginkatu 14-16, SF-O0100 Helsinki, Finland

Abstract: The Association of Finnish Cities (AFC), an advisory organization for all cities and townships in

Finland, had been approached by several cities to recommend a suitable text-processing system. The purpose of this study was to assist the AFC to make a general recommendation for text-processing systems most suitable for Finnish cities. The problem incorporates qualitative and interdependent criteria; we have solved it using Korhonen's hierarchical interactive method published in EJOR, 1986. Keywords: Multiple criteria decision making, qualitative decision problems, subjective evaluations, textprocessing

1. Introduction

The field of office automation has developed rapidly. In the 1980s text-processing systems have become very popular in the administration of Finnish cities. By the beginning of 1985, fifteen different computer hardware suppliers had sold equipment to 72 Finnish cities. Because of the large variety of systems available, it is not easy for a city administrator to decide which textprocessing system to acquire. In 1985, the Association of Finnish Cities (AFC), an advisory organization for all cities and townships in Finland, was approached by several cities to provide recommendations regarding the best or the most suitable text-processing system for them. The AFC decided to make a ranking of the systems currently in use in Finnish cities. In order to develop a general * Communicated by Jyrki Wallenius. Received June 1989

recommendation for systems most suitable for Finnish cities, the AFC collected information on the properties of these systems and their impact on the productivity of the office workers. Accordingly, the development department of the AFC sent a questionnaire to several cities. Our role was to assist the AFC to develop a general recommendation concerning the textprocessing systems most suitable for Finnish cities. We solved the problem by 'a hierarchical interactive method for ranking alternatives with multiple qualitative criteria' (Korhonen, 1986). This paper consists of six sections and two appendices. In Section 2 we describe the problem. Korhonen's method is briefly reviewed in Section 3. We present the analysis and the results of the study in Sections 4 and 5, respectively. Section 6 concludes the paper. In the first appendix we describe the structure of the hierarchy and in the second appendix we present a summary of the rankings of the systems.

0377-2217/91/$03.50 © 1991 - Elsevier Science Publishers B.V. (North-Holland)

180

L. Tanner / Selecting a text-processing system as a qualitative multiple criteria problem

2. Problem formulation

Step 6. Developing the hierarchy. A decisionmaker from the AFC built a hierarchy of criteria based on the questionnaire. The basic structure of the hierarchy is shown in Figure 1 and illustrated in detail in Appendix 1. Step 7. Using the method. Our decision-maker used Korhonen's method (under the name H I R M U ) interactively on a microcomputer. Step 8. Final ranking. The AFC generated the final ranking of the text-processing systems as a result of our analysis and decided to implement the results. Step 9. Publishing the recommendation. The AFC published the results as a recommendation (Ojala, 1985) distributed to Finnish cities needing advice in choosing the most suitable text-processing system to buy.

The AFC wanted to develop a ranking of the text-processing systems most suitable for the administrative offices of Finnish cities.

2.1. General structure Briefly, the steps of our analysis were as follows. Additional comments follow.

Step 1. The need for a general recommendation. Several Finnish cities had asked the AFC for help in selecting the most suitable text-processing system for their administrative offices. Step 2. The questionnaire. In order to receive more information about different text-processing systems from their users, the AFC sent out a questionnaire to the cities which had already bought a text-processing system or several such systems. The aim of the questionnaire was to get feedback from the users of these systems. Step 3. Preliminary results. The AFC analyzed the answers to the questionnaire using basic statistical methods, but they proved to be inadequate for providing a ranking of the systems.

Questionnaire The AFC sent a questionnaire to several cities. Responses were obtained from 147 users from 62 Finnish cities regarding nine different textprocessing systems. The responses were divided into two groups according to the experience of the users in using different text-processing systems. One group consisted of expert typists with experience in using several different text-processing systems and the other consisted of 'novice' typists with experience of one system only. There were 49 responses from the experts and 98 responses from the novices.

Step 4. The AFC solicited our help. Step 5. Choosing the method. We suggested that the AFC use Korhonen's (1986) 'hierarchical interactive method for ranking alternatives with multiple criteria' as the most suitable method for ranking qualitative data.

Levels :

[

~

Properties processing

MOST S U I T ~

of the text systems

* *

TE~-PROCESSING

SYST~]

Impacts of the text-processing systems on the productivity of I the office workers

,I

51

01

7 II

III

or1- I

The levels of the hierarchy: I The ultimate goal (the highest level). II The criteria. III The subcriteria (the lowest level).

Figure 1. The basic structure of the hierarchy

L. Tanner / Selecting a text-processing system as a qualitative multiple criteria problem

The questionnaire consisted of 13 questions regarding the properties of the systems and 17 statements about the impact of the systems on the productivity of the workers. For the first 13 questions, the typists were asked to use a scale from 4 to 10 (4 denotes the worst and 10 the best). For the next 17 statements, a five-point scale was used (2 for total agreement and - 2 for total disagreement). In order to have appropriate data for further analysis, the weighted means were calculated for each system on every question.

Hierarchy Our decision-maker (i.e., the person responsible for this project in the development department of the AFC) built a hierarchy of criteria from the questions and the statements. The whole hierarchy is described in Appendix 1. The goal of the hierarchy is to choose 'The most suitable textprocessing system'. The criteria of the higher level are: (1) Keyboard and display. (2) Editing functions. (3) Search functions. (4) Productivity. (5) Human factors. (6) Service improvement. (7) Change needs. The first three criteria that represent the properties of the systems were further divided into 13 different subcriteria at the lower level. The last four criteria, regarding the impact of the systems on the productivity of the workers, consisted of 17 subcriteria. In the hierarchy, there were 30 subcriteria in total. Alternatives The systems currently in use in Finnish cities were the alternatives. The list of the systems was collected from the questionnaires. They are: Rank Xerox, CPT, Philips, Nixdorf, Esselte, Olivetti, Wang, IBM and Microcomputers & general computers. Having collected the data, the AFC wanted to develop a preference ranking of the systems.

181

ing such problems. For example, the regimemethod (Hinloopen, Nijkamp and Rietvald, 1983), Q-analysis (Duckstein and Kempf, 1981), Roy's outranking-method (Roy, 1973 and Crama and Hansen, 1983), multidimensional scaling (Kruskal and Wish, 1978), lexicographic ordering (Hwang and Masud, 1979), and the analytic hierarchy process (AHP, Saaty, 1980). However, most of the methods do not explicitly take into consideration or deal with the concept of dependencies between the criteria. Therefore, the final rankings obtained by using these methods do not necessarily reflect the 'right' ranking derived from the evaluations of the decision-maker. Next, we demonstrate with a simple example, how dependencies between the criteria may affect the ranking of the alternatives. Suppose that Miss X was struggling with the problem of choosing a spouse. She had three candidates to marry: Mr A, Mr B and Mr C. There were five criteria that she thought to be important to consider: (1) Washing. (2) Cleaning. (3) Cooking. (4) Ironing. (5) Sex appeal. Accordingly, she gave the following weights to the criteria: 1 (Washing), 2 (Cleaning), 1 (Cooking), 2 (Ironing) and 10 (Sex). After she had got to know each of these candidates she described them on a 10-point scale (1 denotes the worst and 10 the best) in Table 1. In her opinion, Mr C was seen as a 'home husband' or 'home father' while Mr A was seen as a 'macho man'. Mr B was something between these two, a kind of 'medium' husband, and still, quite good at everything. Having calculated this linear model she obtained as a result the following numbers for the Table 1 Miss X ' s subjective evaluations of the candidates and her weights for the criteria Candi-

Criteria - weights

2.2 Choosing the method

dates

1 Washing

2 Cleaning

1 Cooking

2 Ironing

10 Sex

The problem was seen as a qualitative multiple criteria decision problem. Several methods have been developed to help the decision-maker in solv-

Mr A Mr B Mr C

1 7 10

1 7 10

1 7 10

1 7 10

10 7 1

182

I~ Tanner / Selecting a text-processing system as a qualitative multiple criteria problem

candidates: Mr A 106, Mr B 112 and Mr C 70. Mr B was considered the 'best' candidate for Miss X to marry even though she had put a lot of weight to sex appeal and had not emphasized household matters (probably she has enough servants at home). The reason why Mr A did not get the highest value lies in the dependencies between the first four criteria. These criteria describe quite similar things ('household' attributes) and therefore, they overlap. The dependent criteria may cause a rank reversal, and a 'false' solution may be implemented. In many decision problems, dependencies are not as obvious and easily seen as in this example. Therefore, we felt that the dependencies could have an important effect on the final ranking of the AFC's problem and we decided to use the 'hierarchical interactive method for ranking alternatives with multiple qualitative criteria' (Korhonen, 1986) to develop the ranking for the systems. In this method dependencies are taken into account by using correlations between the criteria. The criteria are treated hierarchically and the evaluations of the alternatives are based on the subjective judgments of the decision-maker who uses the method in an interactive way.

3. The hierarchical interactive method for ranking alternatives with multiple qualitative criteria Korhonen (1986) introduced a new way to determine the most preferred ranking for a set of alternatives with a large number of criteria in a hierarchical structure. The basic idea is to try to find the most preferred linear combination of the basic criteria, and to maximize the correlations

between these criteria and the linear combination as follows: maximize subject to

u = Rb

b ' R b = 1,

(1)

b>~0. The vector u is an objective function vector to be maximized and R is a non-singular correlation matrix. Then b = R - l u , where vector b is a weight vector for the criteria. Korhonen's method uses correlations between the criteria to deal with dependencies. The structure of the criteria is hierarchical and the criteria may be qualitative or quantitative. The decisionmaker evaluates the alternatives subjectively and the correlations are calculated from these subjective judgments. A preliminary/test program of Korhonen's method has been implemented on S t e p / o n e and I B M / p c m i c r o c o m p u t e r s u n d e r the n a m e HIRMU. The use of colors on the screen helps the decision-maker find the most preferred solution interactively. We now demonstrate the approach in a stepby-step manner by using one part of the AFC's problem as an illustrative example. Let us concentrate on a submodel of the hierarchy (see Appendix 1) where the relevant criterion is number 6: 'Service improvement', with subcriteria: (1) Results and quality of the work. (2) Quality of customer services. (3) Organizational communication. We use the following six text-processing systems as alternatives in this example: Rank Xerox, CPT, Philips, Nixdorf, Wang and Microcomputers. Next the steps of the method are described as implemented in the computer program (HIRMU).

Table 2 The rankings of the systems under each criterion Criteria

(1) Results and quality of the work (2) Qualityof customer services (3) Organizationalcommunication

Text-processingsystems Rank CPT Xerox 1.8 1.8 0.5 0.5 0.5 0.7

Philips

Nixdorf

Wang

Micros

1.8 1.3 0.7

1.6 0.3 0.2

1.2 0.2 0.7

1.3 0.8 0.5

L Tanner / Selecting a text.processing system as a qualitative multiple criteria problem

Step 1. (a) Ask the decision-maker to evaluate the alternatives using all basic criteria by making pairwise comparisons, or by ranking alternatives on an ordinal scale, or by giving strength of preference information on a cardinal scale. Table 2 lists a set of three criteria with the average ranks assigned by the decision-maker to the six text-processing systems. The ranks are given on an ordinal scale from - 2 (total disagreement) to + 2 (total agreement). (b) Compute a correlation matrix R between the basic criteria. Step 2. Compute the linear combination with equal weights as follows: Find b = e / ( e ' R e ) l/2, where e = (1, 1 .... ,1)' and compute u = Rb. Vector u measures the importance of the criteria. The process will generate a feasible (efficient) solution. The solution u = Rb of the problem (1) is efficient (nondominated, Pareto optimal) iff u e C = ((u I u = Rb, b >_ 0). For an3~ proof, see Korhonen (1984). Step 3. (a) Present the initial u to the decisionmaker and ask him to indicate which values he would like to increase. In this example, the initial 'modified correlation coefficients' are presented in Table 3. In order to facilitate the understanding of the u-values, we scaled them to vary from 0 to + 1 (instead o f - 1 to +1). Assume that the decision-maker wants to improve the value of criterion number 1 ('Results and quality of the work'). (b) Construct a reference direction vector Au, such that 1 denotes an increase in the respective criterion value and 0 denotes no such increase. (In the example, the reference direction vector Au is (1, 0, 0)'.) Step 4. Using the reference direction vector (that reflects the desired changes in the criterion values) compute the efficient curve by projecting it on the feasible set.

Table 3 The initial u-values (scaled from 0 to + 1) Criteria

.-values

(1) Results and quality of the work (2) Quality of customer services (3) Organizational communication

0.869 0.886 0.796

Results 0.957

Custome 9.797

183

In£o~Ma 9.729

I

~OHJAUS~ --)

fl

Figure 2. A graphical display of the efficient curve

Step 5. (a) Present graphically to the decisionmaker the efficient curve (Figure 2). (b) Ask the decision-maker to choose the most preferred solution along the efficient curve. The values of the criteria along the efficient curve are plotted on the screen using distinct colors. The decision-maker can examine different solutions on that curve by moving the cursor; the corresponding numerical values of the criteria are displayed simultaneously on top of the screen. In the example, let the most preferred solution along the initial direction be u = (0.957, 0.797, 0.729)' (see Figure 2). (c) If the decision-maker wants to consider other directions, compute a new reference direction vector and return to Step 4. For example, suppose that the decision-maker still wishes to improve the first two criteria ((1) Results and quality of the work, and (2) Quality of customer services). Therefore, at the next iteration he sets Au = (1, 1, 0)'. Now, he chooses u = (0.965, 0.853, 0.659)' as a final solution. The corresponding weight vector is b = (0.763, 0.321, 0.200)'. (d) Consider the next set of criteria and return to Step 2. Or, if all groups of criteria at the basic (or previous) level have been aggregated, compute a new correlation matrix using the weight vectors obtained from the previous level, and return to Step 2. Step 6. (a) If we have reached the top level of the criterion hierarchy, we find the most preferred strength of preference matrix which is then used to obtain the final ranking of the alternatives by computing the row sums of that matrix. In case of

184

h Tanner / Selecting a text-processing system as a qualitative multiple criteria problem

quantitative criteria, these row sums determine an exact ranking for the alternatives, but otherwise, the row sums give an approximate ranking. The Bowman and Colantoni (1973) model would produce an exact ranking in all cases, but solving the model requires a major computational effort even for small problems. (b) Present the final ranking of the alternatives to the decision-maker.

4. Ranking the text-processing systems In order to assist the AFC develop a general recommendation for the text-processing systems most suitable for Finnish cities we used Korhonen's method for further analysis. The AFC collected the data as described in Section 2 and they (i.e., the weighted means) were fed into the computer program (HIRMU) as ordinal data. The weights of the criteria were determined by subjective evaluations of the decision-maker (i.e., the group of some typists in the AFC). 4.1. Subcriteria describing the properties of the systems For criterion (2) Editing functions, the most important subcriteria were: (5) Text editing. (1) Scrolling functions. (3) Search functions. For criterion (3) Search functions, both subcriteria were considered important (i.e., (1) Disk functions and (2) Filing structures). On the other hand, the usability of the keyboard or display (screen) (i.e., criterion (1) Keyboard and display) was considered relatively unimportant. 4.2. Subcriteria describing the impact on the productivity of the office workers Subcriteria (1) Personal abilities and (2) Develop working methods (under criterion (7) Change needs) were highly important. And so were subcriteria (2) Redundant work (under criterion (4) Productivity) and (1) Results and quality of the work (under criterion (6) Service improvement). All the subcriteria measuring human factors were rated low.

By way of comparison, the AFC also wanted to develop rankings using equal weights for the criteria. In solving the problem, the AFC used several different submodels. Firstly, there exist three different hierarchies depending on which criteria are included. One can use the whole hierarchy with all criteria, or the hierarchy consisting of the properties of the systems (first three criteria), or the hierarchy consisting of the productivity aspects (last four criteria). Secondly, there exist three groups of users: all typists, experts, and novices. The experts evaluated six systems: Rank Xerox, CPT, Philips, Nixdorf, Wang and Microcomputers and general computers. The novices used eight systems; the system of Wang had been used by experts only.

5. Results The decision-maker obtained five different rankings for the systems. For the whole hierarchy (i.e., all 7 criteria, all 30 subcriteria, all 9 alternatives, and the weighted means of all users included), the preference ranking of the text processors was the following: (1) IBM. (2) Philips. (3) Esselte, (4) Rank Xerox. Using equal weights for the criteria, the ranking would have been: (1) IBM. (2) Philips. (3) Olivetti. (4) CPT. The text-processing systems (software) of microcomputers and general computers received the lowest scores in both cases. The first submodel consisted of the hierarchy measuring the properties of the text-processing systems (first three criteria, 13 subcriteria, and 6-9 alternatives, depending on the type of users). Firstly, the decision-maker obtained as a result based on the weighted data of all users and nine alternatives the following ranking of the systems: (1) IBM. (2) Olivetti. (3) Wang. (4) Nixdorf.

L Tanner / Selecting a text-processing system as a qualitative multiple criteria problem

The ranking with equal weights was almost the same; only Nixdorf and Esselte changed place. Secondly, the decision-maker ran the model with the data of expert typists and six alternatives. The ranking was: (1) CPT. (2) Philips. (3) Wang. (4) Nixdorf. (5) Micros. (6) Rank Xerox. In the ranking with equal weights Philips got the highest rank and CPT the second highest. Thirdly, the decision-maker used the data of novice typists with eight alternatives. The ranking was the following: (1) IBM. (2) Rank Xerox. (3) Esselte. (4) Olivetti. (5) Nixdorf. (6) CPT. (7) Micros. (8) Philips. Having compared the rankings of these three user groups, we noticed that there was a great difference of opinion between experts and novices. For example, the experts considered Rank Xerox the worst alternative while the novices preferred it over the others. Furthermore, Philips' rank was also reversed. The second submodel consisted of that part of the hierarchy which contained the criteria evaluating the impacts of the text-processing systems on the productivity of the office workers. Now, consider the four last criteria of the total hierarchy and their 17 subcriteria. This model was computed from the data of all users, because there were no differences in the evaluations of experts and novices. The ranking was obtained for all nine alternatives and the most preferred systems were the following: (1) Philips. (2) CPT. (3) IBM. (4) Rank Xerox. The software of micro-computers and general computers was again the least preferred alternative. However, impacts on the productivity of the office workers do not fl'ecessarily depend on a certain text-processing system but the use of a

185

text-processing system may have an effect on the productivity in general. The impacts on the office work were mostly seen as increasing the productivity; the systems reduce overlapping and redundant work, they motivate people to work and to develop working methods. As disadvantages, the typists considered the increased social isolation and the physical and mental strain of the office work. A summary of all the rankings of the systems is presented in Appendix 2. The most important result of this study is the fact that the results of the analysis have been implemented. The AFC published a report (Ojala, 1985) of the rankings. This report serves as a recommendation for cities that are going to acquire such systems.

6. Remarks

Naturally, the report of the AFC and this study will not cover all that is involved in the problem of selecting a text-processing system. The report can be regarded as a recommendation; the rankings are developed according to the subjective assessments of the typists, and therefore, the resuits can be considered as only tentative. In the report, Ojala (1985) proposes that special attention be paid to the expandability and compatibility of a system, the reliability of a supplier and the continuity of activities. Furthermore, the system itself and its 'goodness' are not usually sufficient reasons to buy that system. One should also consider users' guidance, motivation, maintenance, and services of the supplier. These aspects are important as a whole. In this study, the questionnaire was already made before considering the analysis method, and the hierarchy of the criteria was formulated 'artificially' after the data collection. The data was obtained from nine text-processing systems which means that the questionnaire only covered a certain segment of the market. The questions and statements were not unambiguous and the scales used might have an effect on the rankings. Further, some important criteria were missing. For example, the price of the system is usually very important for the buyer, but because of the advisory nature of this study, prices were not taken

L. Tanner

186

/ Selecting a text-processing system as a qualitative multiple criteria problem

into consideration due to changes. Also, maintenance aspects were excluded. The rankings of the systems differed considerably from each other according to the user groups. We cannot explicitly express which group gave the 'best' ranking, or if the grouping has been made correctly at all. There were some rank reversals between experts' and novices' opinions. We may conclude that the 'goodness' and quality of the office automation decisions will become clear after a longer period of use, when the indirect impacts and the versatility of the systems become clear. The decision-maker of the AFC was pleased with the results. He regarded the use of the program as a learning process to gain a better understanding of the entire selection process.

We see that the concept of the dependencies between criteria merits further research. The criteria might be highly interdependent and therefore a rank reversal of the alternatives may occur if some criteria overlap• In the AFC's problem the dependencies were measured by correlations which describe some kind of average dependency between the criteria. However, also other ways of dealing with dependencies should be examined.

Acknowledgements The author wishes to acknowledge the contribution of Professor Pekka Korhonen, Helsinki School of Economics, to this study.

Appendix 1

I PROPERTIES

CHOOSING THE MOST SUITABLE TEXT-PROCESSING

OF THE TEXT-PROCESSING

SYSTEMS

IMPACTS OF THE TEXT-PROCESSING SYST~*{S ON THE PRODUCTIVITY OF THE OFFICE WOP/~ERS

I KEYBOARD & DISPLAY

EDITING FUNCTIONS 1

II

SYSTEM

0 I

5 I

SEARCH FUNCTIONS 1

SERVICE IMPROVEM. ]

5

I

1

•

o oo

m

e" I'-' r~" 0 ~

~,~o

o

~"0

0 ~'~ m ~O~Ct

from

c

~o~