Farm sustainability assessment: some procedural issues

Landscape and Urban Planning 46 (1999) 41±50

Farm sustainability assessment: some procedural issues M. Andreolia,*, R. Rossib, V. Tellarinic b

a University of Pisa, Dip. Economia aziendale,Via C. Ridol® 10, 56124 Pisa, Italy Tuscany Region, Dip. delle Politiche Territoriali e Ambientali, Area Tutela e valorizzazione delle risorse ambientali, Via di Novoli 26, 50127 Florence, Italy c University of Pisa, Dip. Economia dell'Agricoltura dell'Ambiente Agro-forestale e del Territorio, Via del Borghetto 80, 56124, Pisa, Italy

Abstract This article discusses some procedural issues relating to a multicriterial assessment of farm sustainability, based on the criteria proposed by the European Union Concerted Action on `The Landscape and Nature Production Capacity of Sustainable/Organic Types of Agriculture'. Two main problems are stressed: (1) the treatment of basic information used for evaluating farm performances as regards the criteria; and (2) the dif®culties in evaluating a case-study farm. Firstly, the problem of implementing multicriterial analyses when using qualitative ordinal data and discrete quantitative data is faced, stressing the importance of clearly de®ning and applying procedures that can be transferred and repeated. This is due to the fact that almost all research contributions describe in detail multicriterial methods and results, but give little space to the problem of collecting and analysing basic information. Nevertheless, ®nal results heavily depend on the way basic data has been gathered and processed in order to obtain the indices that have been used for the assessment. The lack of standards and of procedure description hampers the comparison of assessments and the possibility to judge their suitability to the aim of farm sustainability assessment. Secondly, the problem of ®nding external points of reference for judging a case-study farm is confronted. Case-studies can be important as `models' for other farms. Indeed, it is easier to persuade farmers to adopt farming styles and decisions that somebody else has already successfully implemented rather than to adopt unexplored ways of managing their farms. This asks for reliable methods to assess a single farm, but almost all multicriterial methods only provide a tool for ranking a set of objects, e.g. farms, from the best to the worst. Conclusions provide some comments on the usefulness of these approaches. # 1999 Elsevier Science B.V. All rights reserved. Keywords: Sustainable farming; Multicriterial analysis; Tuscany; Landscape production; Italy; Case-study assessment

1. Introduction The use of multidimensional approaches, e.g. based on multicriterial analysis, has been a major improve* Corresponding author. Tel.: ‡39-050945315; fax: ‡39050541403. E-mail addresses: [email protected] (M. Andreoli), [email protected] (R. Rossi), [email protected] (V. Tellarini)

0169-2046/99/$20.00 # 1999 Elsevier Science B.V. All rights reserved. PII: S 0 1 6 9 - 2 0 4 6 ( 9 9 ) 0 0 0 4 5 - 6

ment in respect to reductionistic approaches typical of a culture too much based on specialisation (Tellarini et al., paper presented at the EU Concerted Action on Landscape and Nature Production Capacity meeting held in Wageningen, 1996). Studying phenomena from a holistic point of view means taking into account all their relevant facets. Although a holistic approach consents to achieve a full understanding of the phenomena, it asks for tools capable of coping

42

M. Andreoli et al. / Landscape and Urban Planning 46 (1999) 41±50

with multidimensional problems. From an application point of view, the importance of a multidimensional approach in setting up interventions for agriculture is apparent when considering that policies aiming to steer agricultural production or to subsidise farms do not only affect economic and productive results but also affect, e.g. the quality of environment and landscape. The effects of farming on environmental pollution and landscape quality have been studied in Italy, e.g. by Pennacchi, 1994, 1998; Accademia Nazionale di Agricoltura, 1991; Chiusoli, 1994. Policies having only one aim, such as supporting farmers' income as the `old' common agricultural policy (CAP), have often resulted not only in reaching, and sometimes only partially, the intended goal, but they have caused other unforeseen `side-effects'. According to Croci-Angelini (1995), CAP has resulted in deepening regional disparities, while Baldock and Beaufoy (1993) concluded that rationalised intensive agriculture has been associated with damage and destruction of the environment, natural and seminatural habitats and (visual) landscapes. The negative effects that can result from farming have increased the need for sustainable farming practices. A review of the meaning and evolution of sustainability in agriculture has been recently provided by Polinori (1998). A checklist for `Sustainable Landscape Management' has been produced as the ®nal report of the EU Concerted Action on `The Landscape and Nature Production Capacity of Sustainable/Organic Types of Agriculture' (van Mansvelt and van der Lubbe, 1999). This checklist provides an inventory of indices that might be relevant when analysing farming activity impacts. These criteria, ``using a unifying concept derived from Maslow's study on human motivation translated to the landscape and perceived as a re¯ection of the priorities and motivations leading the actions of people'' (van Mansvelt and van der Lubbe, 1999), have been organised in six main ®elds: (1) environment, (2) ecology, (3) economy, (4) sociology, (5) psychology, and (6) physiognomy/cultural geography. Due to the variety of relevant ®elds, sustainable farming has to be analysed using a multidimensional approach. This, however, implies the need to cope with criteria expressed in different units of measurement and with data that are not homogeneous as regard to the level of precision. This asks, ®rst of all, for a

very careful treatment of the data used for building the indices on which to base the ®nal assessment of farm performance, and, secondly, for a rational choice of the methodology to be used for reaching an `overall judgement' (Colorni and Laniado, 1988, 1992). In this context, `overall judgement' indicates a summary of all the performance that the object of the analysis has shown for all the relevant criteria. This article attempts to systemise a series of considerations relating to the above problems, which where stimulated by some of the contributions of the members of the EU Concerted Action on Landscape and Nature Production. 2. The importance of `a priori' clarification of rules and procedures According to Tellarini (1995), in social science empirical research it is possible to distinguish two different phases: the ®rst, called `private phase', which concerns research organisation, data gathering, data veri®cation and data processing; and the second, called `public phase', which involves summarising and commenting results. The ®rst phase is de®ned as private, because it is very seldom fully described by the researcher, since this would take too much space, especially in the case of a multidisciplinary and multicriterial approach. Thus, when presenting multicriterial analyses, quite often only the list of criteria that have been used is provided, without giving any explanation on the way the basic data have been gathered and transformed into indices (e.g. environmental impact criteria in Ciani et al., 1993). According to Colorni and Laniado (1992), the Environmental Impact Assessments performed during the 1980s ``were, in fact, more `surveys' than assessments. Moreover, such `surveys' were performed according to different points of view, with no reference to a common standard: this makes comparison of different studies dif®cult and, even, worse, means that it is often impossible for a public authority to really check the adequacy of the impact study.'' In the same way, the lack of a common standard and of the information needed for fully understanding how criteria have been built does not allow a rational use of many of the studies on the impact of farmers' choices, especially on non-economic parameters. Consequently, although

M. Andreoli et al. / Landscape and Urban Planning 46 (1999) 41±50

the importance of `a priori' clari®cation of rules relating to a scienti®c method may seem an obvious concern, nevertheless, in our opinion, it is important to underline that: The use of qualitative data requires greater attention in the description of hypotheses adopted and of the procedure used for building criteria, since qualitative data are more difficult to interpret objectively than quantitative data. In other words, in our opinion, it is easier to evaluate the difference between a 1000 and a 2000 Euro monthly income than to judge how great is the difference between a good or a normal level of ``offer of sensory qualities, such as colours, smells and sounds.''; Although it is very seldom possible to fully describe in an article the procedures leading to the building of criteria used for an assessment, nevertheless it is necessary that before starting an analysis researchers fully state the procedures for gathering and processing basic information. These procedures should accommodate for the specific requirements of qualitative and quantitative data processing. If, during the analysis, one or several procedures would demonstrate not to be suitable, it is necessary to go back and start over again. Following a stated procedure ensures consistency in data gathering and processing. 2.1. The problem of processing qualitative data When facing a multicriterial analysis, researchers very often have to cope with qualitative variables. Many of the parameters proposed by the EU Concerted Action members for evaluating farm performance (van Mansvelt and van der Lubbe, 1999), such as landscape completeness or wholeness, are qualitative. Moreover, in many cases the cost of quantitative information is so high that, although it might be possible to measure a phenomenon exactly, it is preferable to use a `discrete scale' (e.g. income classes) rather than a continuous scale, or even to use qualitative data, provided that they can be ordered (Andreoli and Tellarini, 1999). In the latter case, researchers have to translate qualitative ordinal information into numerical codes due to the requirements of software for multicriterial analysis. However, researchers should remember that

43

only methods capable to cope with qualitative ordinal data, e.g. concordance absolute index, would give correct results. For building concordance indices it is necessary to compare every possible pair of objects for each criterion and to check if the ®rst has a better, worse or equal performance than the other ones (Colorni and Laniado, 1992). Consequently, this method cannot be used when the analysis is performed on one case-study. The problem of dealing with only a single case-study farm will be discussed later. Let us take the case of erosion in the analysis of two case-study farms (see Rossi et al., 1997; Rossi and Nota, 1999). The erosion analysis was performed by using a ®ve-step scale, since the quality of information was judged insuf®cient for a ®ner scale, where each step was represented by a symbol that was associated to a real situation. The observed situations and associated symbols were the following:     

Clear absence of erosion: ‡‡ Absence of erosion with some uncertainty: ‡ Minimal erosion (without consequences):  Moderate erosion: ÿ Severe erosion: ÿÿ

When transforming qualitative ordinal data into numerical codes and processing them with multicriterial methods, researchers should make sure that: (1) numerical codes are attributed in a rational way, ranking qualitative data, e.g. from the best to the worst, and attributing to them decreasing, or increasing, numerical codes; and (2) the method used for performing multicriterial analysis is suitable for processing qualitative ordinal information, as in the case of concordance absolute index method. In the above-described erosion case (Rossi et al., 1997; Rossi and Nota, 1999), provided that data are considered qualitative ordinal, the translation into numerical codes of the symbols can be done as follows: Symbol Value

‡‡ 1.00

‡ 0.75

 0.50

ÿ 0.25

ÿÿ 0.00

In this case, values have been obtained by giving score `1' to the best situation and score `0' to the worst one and ®nding the three intermediate values in such a way that the scale has a `constant stepping'. This method is very similar to normalisation procedures,

44

M. Andreoli et al. / Landscape and Urban Planning 46 (1999) 41±50

which will be described later. However, when transforming qualitative ordinal data, it is only important that numerical codes can allow ranking situations from the best to the worst, independently from how much a situation differs from the next one. Thus, any scale with decreasing or increasing values can be accepted, independently from the `stepping'. 2.2. Using continuous or discrete quantitative data If in the case of erosion, the above symbols represent a quantitative phenomenon expressed as a discrete scale, the proposed conversion would no longer be correct, insofar as the situation of clear absence of erosion with some uncertainty is much closer to that of clear absence of erosion than to that of minimal erosion (Andreoli et al., 1998). Again, this difference is smaller than that between moderate erosion and severe erosion. In other words, the proposed numerical conversion is correct only if erosion data are processed as qualitative ordinal data. If the initial information is processed as quantitative data, the scale between clear absence of erosion and severe erosion must be divided in a way that more correctly re¯ects the differences in the impact of the erosion levels (Andreoli and Tellarini, 1999). Fig. 1 provides a graphical representation of a possible numerical conversion of the above symbols in the case of qualitative ordinal information

(graphic on the left-hand side) and quantitative information (graphic on the right-hand side). 2.3. From indices expressed in physical units to indices expressed in terms of `Utility' Performing a multicriterial analysis based on continuous quantitative data implies confronting the problem that criteria are expressed in different units of measurement. Measurement units are not relevant if data are qualitative ordinal, because they are used only to compare, for each criterion, if one object of the analysis has a better, worse or equal performance than another. On the contrary, in the case of quantitative data it should be taken into account how much a value differs from another. If all values are transformed into a common unit of measurement, by means of normalisation or other procedures, it is possible to reach an `overall judgement' for every object of analysis by summing up all the values it has scored for the relevant criteria. One of the most common ways for normalising the values of a criterion (Colorni and Laniado, 1988) consists: (a) in giving score 0 to the lowest value observed value in the analysis for that criterion; (b) in giving score 100 to the highest observed in the analysis for that criterion;

Fig. 1. Conversion of symbols relating to real situations into numerical codes, in the case of qualitative ordinal (A) and quantitative discrete (B) data.

M. Andreoli et al. / Landscape and Urban Planning 46 (1999) 41±50

(c) in calculating all the intermediate values by means of a linear transformation. This kind of normalisation has the advantage of always obtaining, for each criterion, positive values ranging from 0 to 100, but it is subject to two main criticisms. First of all, the normalised value given to an object is strictly dependent on which other objects are considered in the analysis; in other words, normalised values for a group of objects for analysis could change if a new object is added or if one of the previous is eliminated from the analysis (Colorni and Laniado, 1988). Secondly, as seen in the above-described example of erosion, very seldom a linear and automatic transformation of values consents to adequately represent differences existing between `real situations'. Conversion of data expressed in physical units into a common measurement unit can also be done by transforming criteria into goals or `objectives' (Colorni and Laniado, 1992). This means expressing criteria in terms of `satisfaction' or `utility' resulting from the physical value of the criterion itself, e.g. evaluating the satisfaction resulting from one, or several, levels of farm incomes or from varying levels of pollutant concentration rather than measuring them in thousands of Euro or in ppm. Thus, rather than transforming criteria in monetary terms, as in the case of cost-bene®ts analysis (Dasguta and Pearce, 1975), the common unit of measure chosen is `Utility'. The concept of Utility is often used in economic analysis, e.g. in describing consumers' behaviour. Indeed, while entrepreneurs are supposed to aim to pro®t maximisation, consumers are supposed to aim at maximising the utility resulting from the consumption of products and services (Samuelson and Nordhaus, 1993). When parameters are expressed in terms of Utility, high values have always a `positive meaning' and low values have a `negative meaning', while this is not true if working with physical units. From this point of view, working with Utility values is easier, because it is not necessary to remember how an index is de®ned (or calculated) for knowing if a high value is desirable, or not. When it is possible to set a target (e.g. an optimal share of fodder crops or a satisfactory level of income) for every parameter, the transformation of conventional data into Utility can be done by giving score `1' when the target is achieved and score `0' when it is

45

not. Since this method provides a too rough measurement scale Ð only two values are allowed Ð it is usually necessary to ®nd an alternative procedure. When quantitative continuous physical data are available, it is possible to have a Utility function that is continuous, rather than dichotomous. Given that the relationship between physical and utility values is very seldom linear, it is necessary to de®ne it case by case. Between the concentration of a pollutant in ppm (parameter in physical terms) and the Utility associated with it, for example, there is an inverse relationship so that as pollution increases, Utility decreases. This relationship is not linear, since it is assumed that the level of pollution has no negative effects on the environment, as long as it is very limited. As the pollutant concentration increases, the quality of the environment worsens, at ®rst quite slowly and then ever more rapidly. In other cases, for example when the density of a natural population is involved, there is no consistently positive or negative relationship between the physical parameter (e.g. expressed as number of animal/ha) and the Utility value. When the density is low its increase determines an increase in Utility, in that the species is reaching optimum density levels; then there is a range of optimum density within which the Utility function maintains its maximum level, but beyond which the satisfaction level decreases again (Andreoli and Tellarini, 1999). The use of Utility function could be criticised in so far as there could be subjectivity in building them. As Bosshard (1997) states ``experiences in landscape planning, especially in the last few years, con®rm epistemological consideration, viz. that a model for evaluating cannot be `objective' Ð in the sense of being generally valid. Rather, every validation is individually dependent on at least the following three premises:  temporary, culturally dependent ideas of values;  the prevailing physical situation;  the personal standpoint of the participants, including that of the experts, with respect to the presentation of the problem.'' This statement does not only apply to the problem of building Utility functions for parameters, but above all affects the problem of deciding the relative importance (weight) to be given to each criterion in comparison to the other ones. Subjectivity in transforming

46

M. Andreoli et al. / Landscape and Urban Planning 46 (1999) 41±50

physical values in satisfaction values Ð as the importance given to each criterion Ð could be limited by applying a procedure capable to accommodate for these causes of variability. In other words, in our opinion, a `satisfactory' level of objectivity and comparability of results might be reached, if rational procedures and benchmarks for transforming physical values in utility values are de®ned. In the same way, although weighting is a subjective process, it is possible to limit its subjectivity by giving guidelines and a rational procedure for attributing weights. A method for attributing weights taking into account the features of impacts (temporary/permanent, local/national, short/long term) and impacted resources (renewable/ not renewable, common/rare, strategic/not strategic) is proposed in Schmidt di Friedberg (1987). Finally, only an exact knowledge of the hypotheses on which data conversion in Utility values and weighting of criteria have been performed can allow readers to judge on the reliability of an analysis. Indeed, the quality of results of an assessment does not depend only on the methodology used for the evaluation, but heavily depends also on the way data used for the assessment have been obtained. 3. Analysing a single case-study Analysing a single case-study is in some way more dif®cult than analysing a set of objects, insofar as it is not possible to perform comparisons between objects. Thus, analysing a single case-study does not allow using qualitative ordinal data, because there is no suitable object to compare data with. Moreover, this means that it is not possible to normalise values due to the lack of internal reference points. Indeed, having only one value for each criterion (the one of the casestudy), the concepts of minimum, maximum and average no longer have any meaning. Consequently, when analysing only one case-study, the transformation of criteria into a common unit of measurement has to be done by means of Utility functions. This, because Utility functions are (or could be) based on external reference points. Due to the fact that Utility data have to be used as quantitative ones, the conversion from physical to Utility units has to be done very carefully. Thus, in our opinion, the conversion should start by de®ning a procedure that:

 sets external points of reference for the minimum and maximum values of the scale, namely the physical situations that correspond to value `0' and value `1' of the Utility function. This process is similar to the one of calibrating a thermometer scale, where value 0 is given to the situation of melting ice and value 100 is given to the situation of boiling water. Varying benchmarks should/could be used for every region. Indeed, according to Hendriks et al. (1999), external reference values may or must differ for different landscape types/regions; since an external point of reference cannot be global, but it must be filled in regionally (see also Rossi et al., 1997). A Utopic region is needed as guiding image for farm development;  does not apply automatic conversions, implying a linear transformation of data, but it tries to define values that are representative of differences in satisfaction relating to real situations. From this point of view, if it is not possible to reconstruct the whole Utility function, it suffices to be able to find the Utility level attributable to the case-study. It is important to note that what is stated as regards conversion procedures, is not only valid for the analysis of a single case-study farm, since the same principles can be adopted when a set of objects are analysed. In fact, while assessing one farm it is only necessary to place a single value in the range de®ned by the 0±1 external points of reference; in the case of a set of objects, there is a number of values to be transformed that correspond to the number of objects. In both cases, in our opinion, it is important to discuss the way external reference points could be chosen. Individuating the values against which to calibrate the scale means deciding which situations to use as references for the maximum and minimum points on the scale. An `objective procedure' for individuating reference points could be the one of taking the best situation achievable in the long term for each parameter as a reference for the maximum Utility. In this case, the term of comparison for judging a case-study would be a `Utopic farm'. The Utopic condition is not so much tied to the achievement of a predetermined maximum target for a single parameter (which might actually be possible for real farms), as to the possibility of reaching the maximum value of all indicators contemporarily. Indeed the concept of Utopic Farm is similar to the one of an `Ideal Point', often used in the

M. Andreoli et al. / Landscape and Urban Planning 46 (1999) 41±50

case of multiple criteria analysis (see, e.g. Romero and Rehman, 1989), which is characterised by the contemporary achievement of all the optimum values (individually achievable) for con¯icting objectives. Using Utopic values as reference points allows for differences due to the speci®c region under examination insofar as it is possible to refer to a situation that expresses the absolute maximum possible of that parameter, independently of the area where the case-study is located (absolute or general Utopia), or to refer to a relative maximum, expressing the maximum level actually possible in that particular region (relative or local Utopia). The choice of a local (or relative) Utopia or a general (or absolute) Utopia conditions the reading of the results as well as the possibilities of comparison when evaluations of different situations are required. So, whereas evaluations expressed against the standard of a general Utopia are directly comparable, since they use the same scale, those expressed according to the standard of a local Utopia indicate the position of the farm with respect to the maximum result obtainable in the reference region, so that the scale is calibrated with a maximum value that varies according to context. Since it is the whole performance, and not one regarding a single parameter, that indicates how much the case-study farm differs from a Utopic farm, it is important to describe how this `overall judgement' on farm performance can be reached. The easiest way of doing it is to sum up all the Utility values scored by each farm, after multiplying them for their weights. In this case each weight represents the relative importance given to a criterion in comparison with the other ones. It should be noted that some researchers are against weighting criteria because weights are the result of ``a subjective, uncertain and con¯ictual operation'' (Colorni and Laniado, 1992) and, consequently, they might be unreliable. However, not using weights when summing up the performance scored for criteria means giving to all of them the same weight, i.e. weight 1; this is again a subjective decision and probably less correct than explicitly giving weights. In this context, in our opinion, it is more suitable to try to control subjectivity, e.g. by giving guidelines for weight attribution (as in Schmidt di Friedberg, 1987) or by checking how much the results of the analysis are dependent on the chosen set of weights, than avoiding using them. In other words, if subjec-

47

tivity is unavoidable, it is at least possible to try to control it and to explicitly state the hypothesis that can be considered subjective in order to make the analysis as `transparent' as possible (Colorni and Laniado, 1992). Since weights are strictly dependent on the socio-economic and environmental context where the analysis is placed, it is not possible to ®nd a weighting system that could be generally valid in every situation. It is apparent, for instance, that developing countries where people still suffer from starvation are more concerned with productive problems of agriculture than with those of landscape preservation. On the contrary, in `rich' countries, environment and landscape are given an increasing interest, in comparison with the problem of agricultural production, which nowadays is often higher than needed. Thus, if a situation implies a level for a criterion which is below the minimum required, nobody would be ready to compensate a decrease in this criterion with an increase in another one, which is less important or which currently has a satisfactory level. Once these physical survival requirements, or needs considered strictly necessary, have been met, it is possible to `trade' between criteria, exchanging the `surplus' of a criterion for an increase in another one. Thus, the trade-off between objectives (represented by criteria) heavily depends on their initial values. Indeed, according to a marginalistic approach (Samuelson and Nordhaus, 1993) usually the importance of an improvement in a criterion is increasingly lower when passing from a mere matching of requirements to increasing levels of surpluses. The above statement is, in our opinion, perfectly coherent with the Maslow's approach to human motivation used as a unifying concept in the EU Concerted Action on The Landscape and Nature Production Capacity of Organic/Sustainable Types of Agriculture (van Mansvelt, 1997; van Mansvelt and van der Lubbe, 1999; Stobbelaar and van Mansvelt, 1999). The use of weighted sum as a method for assessing the overall farm performance and of score 1 as the maximum Utility value results in giving to the Utopic farm an overall judgement of 1. This, because weights are recalculated in such a way that their sum is always 1. Thus, the performance scored by a case-study farm should be read taking into account that the maximum possible level of the overall judgement (i.e. the one of the Utopic farm) is 1. In other words, if a real case-

48

M. Andreoli et al. / Landscape and Urban Planning 46 (1999) 41±50

study farm would have an overall judgement of 0.78, this would mean that its performance is 78% (0.78/1) of the maximum possible, namely the overall judgement of the Utopic farm. It should, however, be remembered that, as shown above, exactly de®ning what Utopia is can be problematic, especially as regards the choice of whether to take as reference the maximum values possible for the various parameters (not always easy to establish) or those that can be considered maximum in the examined context. Indeed, while the Utopic value for the erosion parameter might be objectively generalised in a `clear absence of erosion', this is not the case for parameters such as farm income, where Utopia might be characterised by extremely high values, completely incongruent with the context of the farm under study. To set the external reference for 0 score could be still harder, since using a `too bad' external reference point for score 0 might result in underestimating differences between the other situations. Moreover, the distance between actual farm and Utopia depends on the units of measurement adopted, or rather, on the weighting system used. In other words, when different vectors of weights are used, the distance of a case-study farm from Utopia or `perfection' may vary considerably. Finally, it is important to remember that Utopia is, by de®nition, Pareto-dominant on all the actual or potential farm situations. ``A Pareto optimal solution is a feasible solution for which an increase in the value of one criterion can only be achieved by degrading the value of at least one other criterion'' (Romero and Rehman, 1989). Consequently, a situation is Paretodominant when it is not worse for all parameters and better for at least one. Since Utopia is characterised by scoring the maximum value for each criterion, this means that real farms could match its performance, but not perform better. Thus, Utopia could not be used as a `second object of analysis' for performing a multicriterial analysis based on qualitative ordinal data. Since farmers could consider the Utopic performance to be `out of reach', researchers could consider using a reference point that is closer to the real casestudy situation. From this point of view, another way of calibrating the scale could be the one of using as reference points targets that could be achieved by the case-study farm in the short or long run. In this way, the judgement would consist in an assessment of what the performance of the farm is in comparison with its

potential performance in the long or short run. In other words, with this kind of approach, it could be possible to judge how much ef®cient a farm is; the `inef®ciency' being de®ned as the distance between the case-study farm real situation and its potentiality. Here too, it is essential to understand the type of reference to be used as external term of comparison, a problem that, as in the previous case, brings us back to that of the calibration of the scale. The use of a potential value rather than a Utopic one leads, however, to even greater problems of de®nition, depending on which of the following courses is chosen:  To consider the case-study farm as a homogeneous part of the region in which it is located. In this case, the `local Utopia' could be used as the term of external comparison, i.e. the best performance theoretically obtainable in that context. This course is open to two main criticisms. Firstly, the potentiality of the farm is not necessarily that of the surrounding territory. Indeed, with regard to economic performance, e.g. if the size of the farm is atypical of the area, farm actual potentiality could be quite different from that of the surrounding farms. Secondly, that reference is still made to a Utopic rather than to a potential situation in that account is not taken of the fact that the various objectives are conflicting. In other words, the maximum potential value obtainable for an individual parameter might coincide with the Utopic one, insofar as Utopia refers to the contemporary achievement of the maximum value for all parameters. Thus, by trying to include in `potentiality' the concept of conflicting objectives, it is much more difficult to individuate the set of maximum values for the various objectives that may be contemporarily reached. When analysing a set of farms, a possible way of calculating farm potential in a homogeneous context could be that of considering a select case-study as a benchmark for comparison, after checking that farms under study have the chance of performing as well as the casestudy farm. Although this `applied' potentiality might solve the problem of finding a set of reference values, nevertheless it might underestimate the `theoretical' potentiality. Despite possible criticisms, the local Utopia approach is easy to apply and extends to other farms insofar as it does not ask

M. Andreoli et al. / Landscape and Urban Planning 46 (1999) 41±50

for repeating a double evaluation for them all, i.e. actual and potential situations. However, the adjective `potential' might be misleading since, as we have seen, it is more a question of local Utopia (or of comparison with case-studies) rather than the specific potentials of the farm under study.  To consider the real potentialities of the farm under examination, that need not necessarily coincide with those of the surrounding territory for all parameters. The application of this type of approach involves two rather difficult problems. First of all, it involves the need to carry out a double evaluation, one of the actual situation and another of the potential situation of the farm. In other words, unlike the case of Utopia with its common reference scale for the whole area, here the potentiality of the farm is considered to be specific of the farm itself. Secondly, as in the previous case, the difficulty of defining the potentialities of a farm with regard to a series of criteria relative to objectives that cannot be pursued contemporarily. So, unlike analyses in which only one parameter is evaluated, here there might not be just one but many potential situations depending on the priority given to the achievement of the various objectives. This results in great difficulty in the individuation of the potential situation to be taken as a referent. Moreover, unlike the previous case, it is not possible to use case-studies as external references insofar as farm features are not similar to the one in context. In conclusion, we believe that it is much more dif®cult to determine the margin of improvement of overall farm ef®ciency by making use of targets potentially achievable by the actual farm than to individuate the distance from a situation of local Utopia, even if the former method is formally more correct. This results from the above-mentioned fact that the main difference between a potential situation and Utopia consists in not being capable of pursuing and achieving contemporarily an excellent evaluation for con¯icting objectives. 4. Concluding remarks Performing a farm sustainability assessment is not an easy task, especially if all the relevant effects of farmers' choices have to be taken into account. From

49

this point of view, although the increasing interest of researchers and the whole society are bringing about many studies on this topic, there is still a long way to go. The checklist of criteria proposed by the EU Concerted Action on Landscape and Nature Production constitutes a ®rst step in this direction, providing an inventory of criteria that could be relevant for farm sustainability assessment. Of course, since the checklist is supposed to be valid at the European level, researchers have to select every time which criteria to use and which ones are not suitable for an analysis performed in a speci®c context. The second step that should be undertaken is providing guidelines and standards for using the criteria. This involves two different sets of problems. Firstly, the framework provided by the EU Concerted Action members for sustainability assessment is quite complex. Thus, even if this approach guarantees the reliability of results, nevertheless it asks for a very expensive and timeconsuming data gathering. From this point of view, it might be very interesting to have `shortcuts', i.e. simpli®ed procedures for gathering information that guarantees a `satisfactory', although not optimal, level of quality of information while greatly reducing the effort needed for data collection. Secondly, it would be important to have surveys based on standard procedures, capable of providing researchers with the reference points for calibrating criteria scales for a variety of contexts, characterised by speci®c socio-economic, environmental, and other, features. This kind of research is not always greatly appreciated, since it requires a lot of time and effort and then only provides information for further research. In our opinion, however, this kind of research is important since it allows to perform further analyses, whose results can be compared. Moreover, since in the analysis of casestudy farms researchers could more easily be biased from their opinion on the farms they have selected, the use of standard procedures should be strongly advised. This article has confronted some of the issues relating to procedures that could be used for implementing farms assessment using a multicriterial approach, and it has tried to stress the main problems that can cause surveys and analyses not to be reliable or comparable. This with the aim of promoting a discussion leading to the de®nition of standards that could be employed not only in theoretical research, but also in applied research.

50

M. Andreoli et al. / Landscape and Urban Planning 46 (1999) 41±50

Acknowledgements This research has been supported by National Research Council under contributions Nos. 94.00965.CT06 and 95.03251.CT06, and by the University of Pisa. References Accademia Nazionale di Agricoltura, 1991. Agricoltura e Ambiente, Edagricole, Bologna. Andreoli, M., De Simone, A., Rossi, R., Tellarini, V., 1998. Una proposta di percorso per la valutazione di realtaÁ aziendali o comprensoriali in base ad un set di criteri di tipo qualitativo, Il Borghetto, Pisa. Andreoli, M., Tellarini, V., 1999. Farm sustainability evaluation: methodology and practice. Agric. Ecosys. Environ., in press. Baldock, D., Beaufoy, G., 1993. Nature conservation and new directions in the EC common agriculture policy. Report for the Ministry of Agriculture, Nature Management and Fisheries, The Netherlands. Institute for European Environmental Policy, Arnhem and London. Bosshard, A., 1997. What does objectivity mean for analysis, valuation and implementation in agricultural landscape planning? A practical and epistemological approach to the search for sustainability in `agri-culture', Agric. Ecosyst. Environ. 63/ 2,3: 133±143. Ciani, A., Boggia, A., Marinozzi, G., 1993. Metodologie di valutazione di alternative di parchi: il caso del Parco del Nera. Genio Rurale 11, 46±54. Chiusoli, A., 1994. La rinaturalizzazione del paesaggio agrario: una esigenza ambientale, culturale e civile. Genio Rurale 4, 42± 51. Colorni, A., Laniado, E., 1988. VISPA, Software Territoriale e Ambientale, CLUP, Milan. Colorni, A., Laniado, E., 1992. SILVIA: a decision support system for environmental impact assessment. In: Colombo, A.G. (Ed.), Environmental Impact Assessment, ECSC, EEC, EAEC, Brussels and Luxembourg, printed in the Netherlands, pp. 167±180. Croci-Angelini, E., 1995. Effectiveness and redistribution of the regional policy in the European Community. In: Sotte, F. (Ed.), The Regional Dimension in Agricultural Economics and Policies. Proceedings of the 40th Seminar of the European Association of Agricultural Economists, Grafica Tiburtina, Rome, pp. 251±274.

Dasguta, K., Pearce, D.W., 1975. Analisi costi benefici, teoria e pratica, ISEDI, Milan. Hendriks, K., Stobbelaar D.J., van Mansvelt J.D., 1999. The appearance of agriculture. Assessment of landscape quality of (organic and conventional) horticultural farms in West-Friesland, Agric. Ecosys. Environm., in press. Pennacchi, F., (Ed.), 1994. La riforma Mac Sharry. Effetti nelle aziende R.I.C.A., CNR-RAISA n. 2224, Quaderni dell'Istituto di Economia e Politica Agraria, Tipografia dell'UniversitaÁ degli Studi di Perugia, Perugia. Pennacchi, F., (Ed.), 1998. SostenibilitaÁ Efficienza e Successo Aziendale. Una valutazione nelle aziende della R.I.C.A., CNRRAISA, Quaderni dell'Istituto di Economia e Politica Agraria, quaderno n. 24, Tipografia dell'UniversitaÁ degli Studi di Perugia, Perugia. Polinori, P., 1998. Agricoltura e SostenibilitaÁ. In: Pennacchi, F. (Ed.), SostenibilitaÁ Efficienza e Successo Aziendale. Una valutazione nelle aziende della R.I.C.A., CNR-RAISA, Quaderni dell'Istituto di Economia e Politica Agraria, quaderno n. 24, Tipografia dell'UniversitaÁ degli Studi di Perugia, Perugia, pp. 3±37. Romero, C., Rehman, T., 1989. Multiple Criteria Analysis for Agricultural Decision, Elsevier, Amsterdam. Rossi, R., Nota, D., Fossi, F., 1997. Landscape and nature production capacity of organic types of agriculture: examples of organic farms in two Tuscan landscapes, Agric. Ecosys. Environm. 63/2,3: 159±171. Rossi, R., Nota, D., 1999. Nature and landscape production potentials of organic types of agriculture: a check of evaluation criteria and parameters in two Tuscan farm-landscapes, Agric. Ecosys. Environm., in press. Samuelson, P.A., Nordhaus, W.D., 1993. Economia (Italian ed. of Economics), 14th ed., Zanichelli, Bologna, 1993. Schmidt di Friedberg, P. (Ed.), 1987. Gli indicatori ambientali, Franco Angeli, Milan. Stobbelaar, D.J., van Mansvelt, J.D., 1999. The process of landscape evaluation. Introduction to the second special Agric. Ecosys. Environm. issue of the Concerted Action: the Landscape and Nature Production Capacity of Organic/Sustainable Types of Agriculture, Agric. Ecosys. Environm., in press. Tellarini, V., 1995. Approcci metodologici, in Cannata, G., (Editor) Aziende e famiglie nella collina e montagna appenniniche. Studi di casi, CNR-RAISA, FrancoAngeli, Milan, pp. 23±38. van Mansvelt, J.D., 1997. An interdisciplinary approach to integrate a range of agro-landscape values as proposed by representatives of various disciplines, Agric. Ecosys. Environm. 63/2,3: 233±250. van Mansvelt, J.D., van der Lubbe, M.J., 1999. Checklist for Sustainable Landscape Management, Elsevier, Amsterdam.