Markov Associativities

This article was downloaded by: [Universitaire De Lausanne] On: 19 September 2013, At: 01:32 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Journal of Quantitative Linguistics Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/njql20

Markov Associativities François Bavaud & Aris Xanthos a

University of Lausanne, Switzerland Published online: 16 Feb 2007.

To cite this article: François Bavaud & Aris Xanthos (2005) Markov Associativities, Journal of Quantitative Linguistics, 12:2-3, 123-137, DOI: 10.1080/09296170500172437 To link to this article: http://dx.doi.org/10.1080/09296170500172437

PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sublicensing, systematic supply, or distribution in any form to anyone is expressly

Downloaded by [Universitaire De Lausanne] at 01:32 19 September 2013

forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions

Journal of Quantitative Linguistics 2005, Vol. 12, No. 2-3, pp. 123 – 137 DOI: 10.1080/09296170500172437

Markov Associativities* Franc¸ois Bavaud and Aris Xanthos

Downloaded by [Universitaire De Lausanne] at 01:32 19 September 2013

University of Lausanne, Switzerland

ABSTRACT Quantifying the concept of co-occurrence and iterated co-occurrence yields indices of similarity between words or between documents. These similarities are associated with a reversible Markov transition matrix, the formal properties of which enable us to define euclidean distances, allowing in turn to perform words-documents correspondence analysis as well as words (or documents) classifications at various co-occurrences orders.

INTRODUCTION Two objects are associated if they co-occur frequently enough in the same contexts. In the statistical analysis of textual data, objects can be words and contexts can be documents; associativity between words can be defined as proportional to the probability to draw one of them in a document, given that this document contains the other. With this conception of associativity, one might for instance expect theorem to be little associated to love (because few documents co-cite them), whereas it should be quite associated to logarithm (due to the contribution of mathematical documents), and love and logarithm should be (almost) not associated. Associativities defined in this way are closely related to the components of a Markov transition matrix W, giving the probability to reach a word starting from another (or from itself); we refer to them as Markov associativities. By construction, Markov associativities constitute similarity indices obeying well-identified mathematical constraints *Address correspondence to: Aris Xanthos, Section de linguistique, Universite´ de Lausanne, CH-1015 Lausanne, Switzerland, Tel: +41 21 692 30 09, Fax: +41 21 692 30 55, E-mail: [email protected] 0929-6174/05/1202-30123$16.00 ª Taylor & Francis Group Ltd.

Downloaded by [Universitaire De Lausanne] at 01:32 19 September 2013

124

F. BAVAUD & A. XANTHOS

(symmetry, non-negativity, non-negative definiteness, normalization). They are in principle applicable to any kind of corpus, the choice and organization of which is nevertheless bound to strongly influence the conclusions which may be drawn from this formalism. Markov associativities can be computed from any words-documents contingency table, giving the number of times njk word j has occurred in document k. By duality, i.e. by transposing the matrix njk, the same formalism can serve to define Markov transitions between documents, that is Markov documents associativities. Also, Markov transition matrices can be iterated, yielding higher-order transition matrices (possessing the same stationary distribution). Thus higher-order Markov associativities can be defined in a straightforward way, and capture the idea of higher-order association between objects through co-occurences of order r, for r = 1, 2, 3, . . . . Markov associativities are non-negative definite, which allows us to define euclidean distances between words. Words can thus be represented by a configuration of coordinates, the low-dimensional projection of which aims at maximizing the expressed inertia. The resulting procedure amounts to a factorial correspondence analysis (FCA), endowed with familiar words-documents duality properties. Alternatively, hierarchical classification can be performed, yielding classes of similar words, the composition of which generally vary with the order of the associativity under consideration.

NOTATIONS AND FORMALISM Consider a corpus made of p documents, containing n tokens in total: njk is the number of words of type j = 1, . . . , m occurring in the k-th document P (k = 1, . . . , p) nj! :¼ Ppk¼1 njk is the absolute frequency of word j n!k :¼ m j¼1 Pnjk is the size of document k n!! ¼ n ¼ j;k njk is the size of the corpus n pj :¼ nj! is the relative frequency of word j rk :¼ nn!k is the relative size of document k n n qjk :¼ nj!jkn!k is the associated independence quotient, that is the ratio of the observed versus expected count under independence; by construction, P P p q ¼ r q j j jk j j jk ¼ 1.

125

MARKOV ASSOCIATIVITIES

Words j and j’ co-occurring in the same documents k = 1, . . . , p are associated, and this basic relationship can be quantified by means of an (m 6 m) Markov transition matrix W ¼ ðwjj0 Þ constructed as follows (Fig. 1): n

Downloaded by [Universitaire De Lausanne] at 01:32 19 September 2013

1. for j given, choose a document k with probability pðkj jÞ ¼ njkj! n0 2. then choose a word j’ in document k with probability pðj0 jkÞ ¼ nj!kk The resulting transition matrix reads wjj0 ¼

p X k¼1

pðkj jÞpðj0 jkÞ ¼

p X njk nj0 k k¼1

nj! n!k

¼

p X

rk qjk qj0 k pj0

k¼1

ð1Þ

and enjoys the following properties: 1. 2. 3. 4.

wjj0 % 0 and wj! ¼ 1, that is W is a Markov transition matrix. P m 1 j¼1 pj wjj0 ¼ pj0 that is p is the stationary distribution for W. 2 pj wjj0 ¼ pj0 wj0 j , that is the Markov chain is reversible. ðrÞ For r = 2, 3, . . . , the r-th iterate W & W & & & W ¼ Wr ¼ ðwjj0 Þ is another transition matrix, defining the iterated chain of order r. Wr is also reversible with stationary distribution p, with asymptotic ðrÞ behaviour limr!1 wjj0 ¼ pj0 independently of the initial word or query j (‘‘memory loss’’).

MARKOV ASSOCIATIVITIES Definition: ðrÞ The Markov associativity sjj0 of order r between words j and j’ is (Fig. 1): ðrÞ

sjj0 :¼

1

ðrÞ

wjj0 pj0

ð2Þ

This distribution is unique if njk is irreducible, that is, not degenerate into two or more components (for instance, one component containing French words only in French documents and another containing German words only in German documents, with no lexical intersection). 2 Reversibility characterizes here the word – word or document – document association, and does not refer to the sequential ordering of words inside documents, of course.

Downloaded by [Universitaire De Lausanne] at 01:32 19 September 2013

126

F. BAVAUD & A. XANTHOS

ðrÞ

Fig. 1. Associativity sjj0 of order r (here r = 3).i

Definition (2) makes words associated at order r = 1 (sjj0 large) if they occur often in the same documents (association of order 1). Higher-order ðrÞ associativities sjj0 result from an iteration of the process: at order r, words ðrÞ j and j’ are considered as more associated than average (sjj0 > 1) if the probability to obtain a member of the pair from the other is greater than ðrÞ ðrÞ the average probability (wjj0 > pj0 , or equivalently wj0 j > pj ). Formally: ðrÞ

1. The (m 6 m) associativity matrix SðrÞ ¼ ðsjj0 Þ is non-negative, symmetric (due to the reversibility of wjj0 ) and normalized to P ðrÞ j pj sjj0 ¼ 1. 2. SðrÞ ¼ SPSP & & & PS, where S = S(r) and P is the diagonal matrix containing the pj. The matrix S, and also S(r), can be shown to be positive semi-definite (p.s.d.), i.e. all the associated eigenvalues are pffiffiffiffiffiffiffiffiffiffi 0 j0 . non-negative. In particular, sjj0 ' sjj sjP d 0 jj 3. Particular cases: (a) S0jj0 ¼ p 0 (b) S1jj0 ¼ k rk qjk qj0 k (c) S1 jj0 ( 1. j

ðrÞ sjj0

thus define similarity indices; however, in contrast to a Associativities ðrÞ well-established although little justified tradition, the self-associativity sjj0 ðrÞ ðrÞ ðrÞ is not equal to smax = 1; one finds instead sjj0 % 1 with sjj 6¼ sj0 j0 in general (this can be justified from the particular form of the transition matrix (2)). Also and by construction, the weighted average associativity between any word j and all the other words j’, itself included, is 1: thus in the picture presented here, the more self-associated a word is, the less associated it is with other, distinct words3. 3

cf. the behaviour of category DETDEMFS in illustration 5 below.

MARKOV ASSOCIATIVITIES

127

ILLUSTRATIONS

Downloaded by [Universitaire De Lausanne] at 01:32 19 September 2013

Illustrations 1 to 4 are kinds of Gedankenexperiments, while illustration 5 constitutes a real (although size-reduced) example. Illustration 1 Consider a pair of words (jj’) occurring exclusively together, such as (jj’) = knick-knack. Then njk = nj’k for all k, and in particular qjk = qj’k for all k; more precisely, the latter identity holds iff the lexical profiles are proportional, that is njk = a nj’k for all k. Then sjj’ = sjj = sj’j’, that is j and pffiffiffiffiffiffiffiffiffiffi j’ are maximally associated in view of the property sjj0 ' sjj sj0 j0 . HigherðrÞ ðrÞ ðrÞ order associativities inherit this property: sjj0 ¼ sjj ¼ sj0 j0 . Illustration 2 British and American English display some regular spelling variations, for instance -our versus -or endings. Although chances that j = colour and j’ = color co-occur in the same document are low, they are likely to be ð1Þ strongly associated with the same words, which results in sjj0 ffi 0 and ð2Þ sjj0 >> 1. Illustration 3 Words {j} such as freedom, free, freed, . . . can bePgrouped into the same supra-category P J, of relative frequency pJ ¼ j2J pj and associated p quotient qJk ¼ j2J pJj qjk . Also, other words {j’} may be grouped into supra-categories J’. The resulting J = 1, . . ., M 5 m supra-categories and the associated (M 6 M) associativity matrix transform as sJJ0 ¼ P P pj pj0 s of non-negativity, symmetry, j2J j0 2J0 pJ pJ0 jj0 . It inherits the propertiesP P ðrÞ p p 0 ðrÞ normalization and p.s.d. However, sJJ0 6¼ j2J j0 2J0 pJj p j 0 sjj0 in general J for r 5 2: words aggregation and Markov iteration do not commute. Illustration 4 Documents {k} can also be concatenated into Psupra-documents K = 1, and quotients . . . ,P P 5 p, of relative sizes rK ¼ k2K rP k qjK ¼ k2K rrk qjk . The resulting dissimilarity s^jj0 ¼ K rK qjK qj0 K is still K non-negative, symmetric, normalized and p.s.d. By Jensen’s inequality, the diagonal associativities decrease under aggregation !2: Xr X X Xr X X k 2 k rk q2jk ¼ rK qjk % rK qjk ¼ rK q2jK ¼ s^jj sjj ¼ r r K K K k k2K K k2K K

128

F. BAVAUD & A. XANTHOS

Downloaded by [Universitaire De Lausanne] at 01:32 19 September 2013

which shows that, on average, off-diagonal associativities increase under aggregation. "One gets s^jj0 ( 1 in the limit of one single document (p = 1), and s^jj0 ¼ djj0 pj in the limit of minimal ‘‘one-token documents’’ (p = n). Illustration 5 In the framework of structural linguistics, it is common to discriminate between syntagmatic and paradigmatic relationships between linguistic units. The first term refers to units co-occurring within a relevant context, while the second corresponds to units which can be substituted to each other in a given context but cannot occur together.4 The following illustration shows that, in the domain of syntax, these different relationships yield specific patterns of r-th order associativity. We systematically extracted nominal phrases out of a French journalistic corpus,5 replacing the actual words by their syntactic category. After sampling, we obtained a corpus of size n = 2’914, containing p = 1’239 phrases (documents) and m = 26 categories (word types).6 After computing the corresponding transition W r and similarity S r matrices for various orders, it turned out that pairs of categories seemed to exhibit mainly three specific relationships: (a)

(b)

4

Some pairs appear to be only lightly similar or dissimilar at order 1, and tend towards the average similarity of 1 as r gro ws (possibly crossing that limit, but never in a significant way). For instance, this is the behaviour of pairs (PREP, ADJFS) or (DETPOSS, NCMP) (Fig. 2). Elements in such pairs have no particular syntactic relationship together. Some other pairs of categories show a high or low first-order similarity, tending to the average similarity as r grows. This behaviour characterizes elements with a strong tendency to cooccur (or not) in phrases, like the pairs (ADJFS, NCFS) or (ADJFS, NCMS), the second of which violates the rule that noun-

A significant exception to this is the case of co-ordination. La Liberte´, edited in Fribourg, Switzerland. 6 Key to the abbreviations: PREP = preposition, ADV = adverb, NC(MjF)(SjP) = masculine/feminine singular/plural common noun, ADJ(MjF)(SjP) = masculine/ feminine singular/plural adjective, ADJ(SjP)IG = idem, gender-invariant, DET(IjDjDEMjPOSS)(MjF)S = indefinite/definite/demonstrative/possessive masculine/ feminine singular article, DET(IjDjDEMjPOSS)(SjP)IG = idem, singular/plural genderinvariant. 5

Downloaded by [Universitaire De Lausanne] at 01:32 19 September 2013

MARKOV ASSOCIATIVITIES

129

Fig. 2. r-th order associativity between unrelated categories.

(c)

adjective groups should possess an unified gender in French (Fig. 3). The last case is that of mutually exclusive elements but liable to ‘‘co-co-occur’’ within the same contexts. Their similarity is minimal for r = 1, as they never occur in the same phrase, but it goes significantly beyond the average for r52 before regressing to it for higher orders. Pairs (DETDMS, DETIMS) and (DETDFS, DETFMS) are prototypical examples of this (Fig. 4).

MARKOV DISSIMILARITIES: FCA AND CLASSIFICATION ðrÞ

Associativities sjj0 are positive semi-definite, and play the role of the ‘‘scalar product matrix’’ in the classical multidimensional scaling problem (see, for example, Schoenberg, 1935; Gower, 1982). Following the latter, ðrÞ we construct embeddable dissimilarities Djj0 of order r as

Downloaded by [Universitaire De Lausanne] at 01:32 19 September 2013

130

F. BAVAUD & A. XANTHOS

Fig. 3. r-th order associativity between syntagmatically related categories.

ðrÞ

ðrÞ

ðrÞ

ðrÞ

Djj0 :¼ sjj þ sj0 j0 + 2 sjj0

ð3Þ

The weighted average dissimilarity between all pairs of words is the inertia of order r defined as X ðrÞ X X ðrÞ 1X ðrÞ ðrÞ IðrÞ :¼ pj pj0 Djj0 ¼ pj sjj + pj pj0 sjj0 ¼ wjj + 1 ð4Þ 2 jj0 j j jj0 Hence, the higher the probability of getting the same word (that is the higher the average self-associativity), the higher the corresponding P inertia.P Particular cases are I(0) = m – 1, Ið1Þ ¼ j pj sjj + 1, Ið2Þ ¼ jj0 pj pj0 sjj0 + 1 and I (1) = 0. The inertia of order r = 1 is nothing but the chi-square (per count) associated to the words-documents contingency table (njk) (see also Bavaud, 2002):

Downloaded by [Universitaire De Lausanne] at 01:32 19 September 2013

MARKOV ASSOCIATIVITIES

131

Fig. 4. r-th order associativity between paradigmatically related categories.

Ið1Þ ¼

X j

pj sjj + 1 ¼

X jk

pj rk q2jk + 1 ¼

1 X ðnjk + npj rk Þ2 w2 ¼ n jk npj rk n

ð5Þ

Factorial correspondence analysis (FCA) aims at representing words j = 1, . . . , m as points xja such that a maximum part of inertia I(1) is expressed by the first dimensions a = 1, 2, . . . , (see, for example, _ Greenacre, 1984). The resulting co-ordinates {xja} constitute a lowdimensional, factorial representation of words, in contrast to the highdimensional, direct representation {xjl} introduced above. Higher-order FCA, generalizing the ordinary FCA of order 1, can be constructed as follows: consider the spectral decomposition CðrÞ ¼ UðrÞ LðrÞ ðUðrÞ Þ0 (with U(r) orthogonal and LðrÞ diagonal with decreasingly ordered values) of the . symmetric (m 6 m) matrix pffiffiffiffi ðrÞ pffiffiffiffiffi ðrÞ ðrÞ CðrÞ ¼ ðcjj0 Þ defined as cjj0 :¼ pj wjj0 pj0 ; identity C(r) = C r entails

Downloaded by [Universitaire De Lausanne] at 01:32 19 September 2013

132

F. BAVAUD & A. XANTHOS

U(r) = U = (uja) (independently of r) and LðrÞ ¼ Lr with diagonal r coordinates are components #pffiffiffirffi "plffiffiaffiffi.$ The searched 2 P ðrÞ for ðrÞ words ðrÞ ðrÞ laP pj uja , obeying a ðxja + xj0 a Þ ¼ Djj0 as requested.7 xja :¼ Also, IðrÞ ¼ a%2 lra , which shows the first non-trivial dimensions a = 2, 3, . . . , to express a maximum part of the projected inertia I(r) (l_ 1 = 1 corresponds to the trivial eigenvalue) (see Fig. 5). As in ordinary FCA, duality enables the same eigenstructure to generate the higher-order factorial representation of documents coordiðrÞ nates yka . Finally, a classification of words can be performed. Figures 6 to 8 show the results of hierarchical Ward classifications applied on (3) with r = 1, 2, 3 respectively. Cutting the dendrograms at some height h (here represented horizontally in arbitrary units) sends the j into Pm individuals ðrÞ p ðrÞ M 4 m groups J, with group coordinates xJl :¼ j2J pJj xjl ; inertia of order r decomposes into a between- and a within-group contribution: X X pj ðrÞ 1X 1X ðrÞ ðrÞ ðrÞ ðrÞ pj pj0 Djj0 ¼ pJ pJ0 DJJ0 þ pJ DjJ ¼: IB þ IW IðrÞ ¼ 2 jj0 2 JJ0 p J j2J J ð6Þ

P ðrÞ ðrÞ ðrÞ where P DjJ :¼ l ðxjl + xJl Þ2 is the word-group dissimilarity and ðrÞ ðrÞ ðrÞ 2 DJJ0 :¼ l ðxJl + xJ0 l Þ the group-group dissimilarity. Under aggregation pj pj0 ðrÞ ðrÞ ðrÞ Djj0 . J; J0 ! ½J [ J0 -, the intra-group inertia IW increases of DIw ¼ pj þp j0 Aggregating groups minimizing this increase (as in Figures 6 to 8) amounts to Ward clustering algorithm (see, for example, Lebart et al., 1995). Interestingly enough, changing the order r ! r0 transforms the FCA representation into another representation which is pretty0 close to the former, since the eigenvalues solely are altered as lra ! lra ; by contrast, the associated classification can be altered fairly more substantially, as attested by Figures 6 to 8.

7

Proof: P

ðrÞ ðrÞ ðx +xj 0 a Þ2 a ja

¼

X a

ðrÞ

ðrÞ

ðrÞ

ðrÞ

ðrÞ ðrÞ cj0 j0 cjj 0 wj 0 j 0 wjj 0 cjj wjj uja uj 0 a lra ðpffiffiffi + pffiffiffiffiffi0ffiÞ2 ¼ þ 0 + 2 pffiffiffiffipffiffiffiffiffi0ffi ¼ þ 0 +2 pj pj pj pj pj 0 pj pj pj pj

ðrÞ

ðrÞ

ðrÞ

ðrÞ

¼ sjj þ sj 0 j 0 + 2sjj 0 ¼ Djj 0

Downloaded by [Universitaire De Lausanne] at 01:32 19 September 2013

MARKOV ASSOCIATIVITIES

133

Fig. 5. FCA scores for words.ii

CONCLUSIONS AND FURTHER ISSUES The present work has explored some formal properties of the concept of associativity of order r, demonstrating how it can be statistically founded and used in a classical data-analytical framework. In the vector space representation of information retrieval (IR) (see, for example, Slaton and Buckley, 1988; Besanc¸on et al., 1999), document – document similarities are typically defined as

Downloaded by [Universitaire De Lausanne] at 01:32 19 September 2013

134

F. BAVAUD & A. XANTHOS

Fig. 6. Ward classification on D1jj0 as defined in (3).

ðak ; ak0 Þ ~kk0 ¼ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ; s ðak ; ak Þðak0 ; ak0 Þ

P where ðak ; ak0 Þ :¼ j akj ak0 j and akj is the vector of terms weights associated with document k, with p akj ¼ ð1 þ log njk Þ log #fkjnjk > 0g if njk 5 0 and ajk = 0 otherwise. By contrast, first-order Markov document-document similarities (2) express as s~kk0 ¼ ðbk ; bk0 Þ, where rffiffiffiffiffi n njk : bkj ¼ nj! n!k ~kk0 , the associativity s~kk0 is invariant under the aggregation Contrarily to s of words possessing identical profiles, as does the generalized family rffiffiffiffiffi % & nj! njk n f bkj ¼ n nj! n!k

Downloaded by [Universitaire De Lausanne] at 01:32 19 September 2013

MARKOV ASSOCIATIVITIES

135

Fig. 7. Ward classification on D2jj0 as defined in (3).

(Bavaud, 2002). In that respect, Markov associativities could play the role of reference similarities, endowed with appealing formal properties, to which the various tf-idf weighting schemes proposed and evaluated in the literature might be compared. Also, the present formalism could be further developed by considering fuzzy memberships and associativities (Bavaud, 2004), or by incorporating work on probabilistic latent semantic analysis (Hofmann, 1999), P postulating conditional probability of the form pðjjkÞ ¼ z pðjjzÞ pðzjkÞ, where z indexes latent classes. Others extensions implying non-linear distortions of distances conserving the euclidean property, trade-off between orders by using chain mixtures, and special documents definitions linking with the n-grams formalism are currently under investigation. Although we are confident about the formal strength of our formalism, which we judge as sound and statistically founded (and obviously not restricted to textual data), results on large-scale and systematic empirical performance of IR systems based upon the present formalism are presently yet missing. This state of things should be remedied in priority: at the time being, the question of whether our formalism will perform

Downloaded by [Universitaire De Lausanne] at 01:32 19 September 2013

136

F. BAVAUD & A. XANTHOS

Fig. 8. Ward classification on D3jj0 as defined in (3).

better in practice that another system based upon somewhat ad hoc assumptions remains open.

ACKNOWLEDGEMENTS Thanks to M. Rajman and J.-C. Chappelier for stimulating discussions, and to N. Jufer and S. Durrer for their textual data.

Notes ðrÞ

i The associativity sjj0 of order r is the ratio of the probability to get the word j’ starting from word j to the relative frequency of word j’ : first, draw a document k containing word j, pick another word l in k, find another document k’ containing l, pick another word l’ in k’, find another document k’’ containing l’, and finally pick (or not) word j’ in k’’. ii Singular-plural factor a = 2 opposes cluster 2 ((ADJjDETPOSS)SIG) to cluster 4 (NC(FjM)P, ADJ(FjM)P, ADJPIG, ADJNUM, DET(DjIjDEMjPOSS)PIG). Masculine-feminine factor a = 3 opposes cluster 1 ((NCjADJ)MS, DET(DjIjDEMjPOSS)MS) to cluster 3 ((NCjADJ)FS, DET(DjIjDEMjPOSS)FS).

MARKOV ASSOCIATIVITIES

137

Downloaded by [Universitaire De Lausanne] at 01:32 19 September 2013

REFERENCES Bavaud, F. (2002). Quotient Dissimilarities, Euclidean Embeddability, and Huygens’ Weak Principle. In K. Jajuga, A. Sokolowski & H.H. Bock (Eds.), Classification, clustering, and data analysis (pp. 195 – 202). Berlin: Springer. Bavaud, F. (2004). On the comparison and representation of fuzzy partitions. To appear in Student. Besanc¸on, R., Rajman M., & Chappelier, J.-C. (1999). Textual similarities based on a distributional approach. In Proceedings of the Tenth International Workshop on Database and Expert Systems Applications, Firenze, Italy (pp. 180 – 184). Gower, J.C. (1982). Euclidean distance geometry. The Mathematical Scientist, 7, 1 – 14. Greenacre, M. J. (1984). Theory and applications of correspondence analysis. New York: Academic Press. Hofmann, T. (1999). Probabilistic latent semantic indexing. In Proceedings of the 22nd International Conference on Research and Development in Information Retrieval, Berkley, USA (pp. 50 – 57). Lebart, L., Morineau, A., & Piron, M. (1995). Statistique exploratoire multidimensionelle. Paris: Dunod. Slaton, G., & Buckley, C. (1988). Term weighting approaches in automatic text retrieval. Information Processing and Management, 24, 513 – 523. Schoenberg, I. J. (1935). Remarks to Maurice Fre´chet’s article ‘‘Sur la de´finition axiomatique d’une classe d’espaces vectoriels distance´s applicables vectoriellement sur l’espace de Hilbert’’. Annals of Mathematics, 36, 724 – 732.