DATA MINING FOR TRACKING GENDER BIAS: THE 2012 SOCIALIST PRIMARIES CONTEST

June 14, 2017 | Autor: Fabienne Baider | Categoria: Gender Discourse, Corpus Linguistics and Discourse Analysis
Share Embed


Descrição do Produto

CHAPTER TWELVE DATA MINING FOR TRACKING GENDER BIAS: THE 2012 SOCIALIST PRIMARIES CONTEST FABIENNE H. BAIDER1 AND ÉVELYNE JACQUEY2 1

2

UNIVERSITY OF CYPRUS, CYPRUS ATILF-CNRS RESEARCH UNIT, FRANCE

Social gender and journalistic political discourse Political scientists such as Coulomb-Gully (2009, 2012), Bonnafous and Coulomb-Gully (2007) and Bertini (2007) have denounced the gender bias of journalistic discourse which, according to media and gender specialists, is built “on women’s backs, digging into this old sexist background, typical of our societies” (Bergvall 1999: 283). Elsewhere, qualitative linguistic research has shown the impact of syntactic structures and lexical choices on our perception of entities (Michard 1988; Macaulay & Brice 1997; Eckert & McConnell-Ginet 2003; Michard & Ribéry 2008), a perception which is ideologically constructed. A case in point is recent linguistic research (Valette & Rastier 2006; Reutenauer et al., 2010; Valette 2010) which suggests that specific lexical and stylistic choices can be found in texts that are considered to be racist. Recurrent linguistic choices appear to construct power relationships between the entities described, transferring ideological prejudices onto these linguistic choices. In previous research (Baider 2009; Baider & Jacquey 2010), we have suggested that working on a journalistic corpus with specific software can help tracking bias in journalistic discourse. In a study focusing on linguistic patterns and choices in a corpus covering the 2007 presidential campaign in France, it was suggested that social gender (i.e. stereotypical expectations about who will be a typical member of a given category) affected representations of leadership in journalistic discourse, with striking asymmetries between female and male candidates. However, this

Data Mining for Tracking Gender Bias

223

conclusion was not corroborated by data involving female and male candidates who had no chance of winning (i.e. who were not frontrunners). We suggested that this specific asymmetry would be true only when the stakes are particularly high, or alternatively only when a female politician specifically plays the ‘woman’ card. Our present work focuses on the French socialist party primaries (October 2011), concentrating on the two front-runners, François Hollande and Martine Aubry. In the case of the 2011 socialist party primaries, two elements are similar to the presidential finalists’ duel between Ségolène Royal and Nicolas Sarkozy in 2007. First, we have a male-female match; and second, the stakes were high since the winner would go on to face Sarkozy, the then president in the second round. However, one marked difference is that Aubry, the female candidate, did not specifically play the gender card. So, if gendered stereotypes were to be found, journalists could then be partly responsible for constructing such a gendered identity.

Theoretical framework Critical Discourse Analysis (CDA) framework In this chapter, we consider language as a form of social practice and we base our analysis on the following three assumptions which underpin work in critical discourse analysis (Fairclough 1989: 22):  language is a part of the society and not somehow external to it;  language is a social process;  language is a socially conditioned process, conditioned that is by other (non-linguistic) parts of society.

CDA, which Fairclough (1995: 132) defines as the systematic exploration of “often opaque relationships of causality and determination between (a) discursive practices, events and texts, and (b) wider social and cultural structures, relations and processes”, can be used to explore the social reality of French politics, French political culture and journalistic practices. The aim of our study is, then, to uncover the implicit ideologies in texts and therefore the exercise of power in texts (cf. Widdowson 2000; van Dijk 2006). The wider macro-context, i.e. where the discourse is set in terms of its social reality, has to be taken into consideration after study of the micro-level, i.e. the text itself. At the micro-level we will consider what previous linguistic analyses on racist and sexist discourse have noted, and in particular we will look at the most frequent and salient collocations,

224

Chapter Twelve

as well as the polarity of discourse (negative-positive) constructed with the choice of verbs, the use of negative adverbs, etc.

Discourse and ideology As we mentioned above, research on journalistic practices (interviewing, reporting speech of politicians, etc.), whether in sociology or political sciences, has revealed a gendered bias, which has been imputed to the journalists. Studies carried out on the political press by SrebernyMohammadi and Rossi (1996), Penelope (1990) and Roncarolo (2000), amongst others, have pointed out different practices when describing female politicians, and that ‘difference’ works to their disadvantage because of the gender stereotypes. The explanation, according to Kanter (1977), would be that as token individuals, women in politics suffer a typical role of ‘entrapment’: when women are numerically underrepresented (as is the case in politics), they are forced to choose between a few stereotypical roles such as the ‘mother’, the ‘pet’, the ‘seductress’ and the ‘iron maiden’, whether these roles are chosen by the female politicians themselves or set up by journalists. However, most studies conclude that the media are more responsible for gender stereotyped coverage of women than any actions on the part of the women themselves (Kahn 1994, 1996; Gingras 1997; Roncarolo 2000; Niven & Zilber 2001). As far as the French press is concerned, Bonnafous and CoulombGully (2007), Bertini (2007), Coulomb-Gully (2009, 2012) and Fracchiola (2011) have all agreed on the critical role played by journalistic discourse in the construction of politicians as being ‘good’ or ‘bad’ actors. As far as sexism and linguistic patterns are concerned, previous studies have shown that in many English-language texts, most of the agents or ‘doers’ are men and boys whereas “women and girls, if present at all, are often themes to which things happen” (Eckert & McConnell-Ginet 2003: 73; see also Penelope 1990; Macaulay & Brice 1997; Lazar 2005). Michard (1988) and Michard and Ribéry (2008) demonstrate a structural asymmetry in the gender order, on the basis of a rigorous enunciativist1 analysis of lexical and grammatical choices made by writers in scientific texts. They show how processes construct, perpetuate and reinforce a hierarchy between entities. Michard and Ribéry’s (2008: 174184) analysis highlights in particular the following: 1

See Culioli (1990) on the enunciativist framework and Groussier (2000) for an overview in English.

Data Mining for Tracking Gender Bias 



225

the lack of agency in discourse of the least favoured entity – association with automatic processes as the ones describing instruments or machines or construction as non-animated because of the choice of verbs or nouns referring to them; a high number of adverbs and adjectives and the use of negative modifiers such as few, a little, a few, less collocated with the noun phrases (NPs) under investigation.

Combining these syntactic and lexical findings, Michard and Ribéry (2008) conclude that a power relationship can be established discursively between the described entities. Other more recent linguistic research in French discourse and ideology has also suggested that lexical and stylistic choices could be specific in text labelled as racist, although the text is not explicitly racist (Valette & Rastier 2006; Reutenauer et al. 2010; Valette 2010). In particular, these researchers mention the presence of the following:  

evaluative adverbs (tout ‘all’, rien ‘nothing’, jamais ‘never’, toujours ‘always’, beaucoup ‘a lot’, trop ‘too much’, etc.); pejoration morphemes (such as -ouille, -âtre) and references to animals or disease as well as lexical amalgams such as Afro-maghrébins ‘AfroMaghrebians’, etc.

Given these earlier studies which delineate linguistic patterns that sustain ideological points of view, we postulate that assessing the lexical environment of each NP under study (Aubry and Hollande) will provide insight into how discursive practices reveal prejudices during the primaries.

Previous research on gender bias in the French Press Our own research has suggested that analysis of discourse practices by focusing on NPs referring to female and male politicians could be a way of assessing the ‘linguistic portrait’ of politicians in journalistic discourse, and in particular their agency. Studying asymmetry in discourse, as recommended by Pauwels (1998) and Cameron (2003), may reveal whether political discourse studies which report discriminatory discourse practices towards women are still founded. In previous research (Baider & Jacquey 2010), we looked at categories of variables (lexical and grammatical choices), and related these to two independent variables: the sex of the politicians and the party they belong to. To study discourse practices in the 2007 presidential contest, we

226

Chapter Twelve

examined a 19,000 word corpus (coverage in three national daily newspapers: Libération, Le Monde and Le Figaro), taking into consideration terms of address, the most commonly used verbs when referring to the politicians, and their syntactic and semantic functions. Using the semantic analysis software SEMY (Grzesitchak et al. 2007; Reutenauer et al. 2010) and the definitions of the Trésor de la langue française informatisée (TLFi), we considered the following: the semantic fields defining each candidate by listing the recurrent semantic content of any word found in the context of the nouns Royal and Sarkozy; and a hierarchy within which semantic content is the most frequent for each candidate. According to these data, the female politician (Royal), in comparison to the male candidate (Sarkozy), was portrayed as more on the defensive than on the attack. She also related to the public in order to stage her ambitions, but not to present a detailed programme. Sarkozy, on the other hand, came across as a fighter, set to put his plans into motion, both important qualities for a future head of state. These lexical choices weakened Royal’s position as presented in the discourse (Baider & Jacquey 2010). We restricted our study to Agent and non-Agent categories (Dowty 1991; Siewierska 1991) in order to examine the actual causality of each NP. We found that the syntactic choices made in journalistic discourse when referring to the presidential candidates supported a partial dichotomy with the proper noun Sarkozy functioning more often as a subject than Royal. The difference was even greater as far as the semantic functions were concerned. This difference was also true with the third male candidate, François Bayrou, whose name presented the same linguistic profile as Sarkozy. However, we did not find this asymmetry with data relating to less central candidates (e.g. Dominique Voynet, José Bové, Marie-George Buffet and Olivier Besancenot). It was suggested that when the stakes are not as high as for a presidential election, reproducing gender stereotypes in the political sphere did not seem to be on the journalists’ agenda. For example, Marie-George Buffet (the then communist party leader) was granted more agency than her young opponent Olivier Besancenot (the then extreme leftwing party leader). Parity can be afforded when there is not much to lose or to gain2. It should also be mentioned that the Sarkozy and Royal candidatures were considered to be somewhat ‘exceptional’ cases since, as French voters and commentators generally noted, the two politicians were playing 2

A tactic which is also followed by some parties when drawing up their list of candidates for elections: women are present in regions where stakes are not high or where they are most likely to lose (Sineau 2010).

Data Mining for Tracking Gender Bias

227

for the first time in French political history on sexual stereotypes (Gingras 1997; Achin & Dorlin 2008; Coulomb-Gully 2009, 2012). Thus, the French press reports specifically on Royal (vs. Sarkozy and Bayrou) during the 2007 presidential campaign may have been driven by (and may have helped to perpetuate) a normative view of the sexes “constructed against the backdrop of strong social stereotypes and ascriptions about gender” (Bergvall 1999: 283). However, these results could be due to the politician herself, and therefore her linguistic profile could also be considered as an exception but not the rule given our other results. In order to test the validity of our previous hypothesis and to decide on the different explanations put forward, we go on here to analyse the socialist primaries.

Corpus and data From a total of six different newspapers (national dailies: Libération, Le Monde, Le Figaro, La Croix; regional dailies Ouest-France, Est Républicain), twenty articles were selected for each of the primary candidates – a total of 120 articles for each candidate. We also gathered all the articles in all these newspapers for the given periods to form a reference corpus. Articles were chosen from periods considered to be the most decisive for the primaries (March-July 2011; September-October 2011).

Extraction of patterns We looked at the most frequent collocations found in the context of the NP under investigation (such as primaire socialiste ‘socialist primary’) and the most specific collocations for that NP (such as tonton apaisant ‘appeasing uncle’). The software TERMOSTAT (Lesage et al. 1993; Drouin 2003) was used to examine both the most frequent and the most specific lexical categories for each NP under study (Aubry and Hollande)3. From a qualitative perspective, KWIC display using ANTCONC allowed us to study contexts for the NP of each candidate. We also looked for the 3

Specificity is calculated by comparing our data to the ‘reference corpus’ (28 million words) from Le Monde (2002). Frequency and specificity are criteria for corpus analysis, as previous studies on ideology and discourse have noted (Michard & Ribéry 2008; Valette & Rastier 2006). Especially since forms such as derogative adverbs and negative adjectives are described as the most relevant to ideological loading (cf. Reutenauer et al. 2010; Valette 2010), it is useful to be able to identify such grammatical categories.

228

Chapter Twelve

discriminatory practices pointed out by Valette (2010) and Michard (1988) which include constructing ‘the other’ as inanimate by the use of comparisons with object4 and embedding the entity within a derogatory discourse (e.g. at a semantic level with the use of negative adverbs, nouns, etc.; and at a morphosyntactic level, using moods and certain modal verbs). In the remainder of this chapter we follow the three dimensions described in Fairclough (1989: 26-27) for presenting the data. First, we describe the data in order to identify the formal properties of the text and patterns in it, this being done with the help of data mining tools such as TERMOSTAT. Next, we interpret the data and identify the relationship between the journalistic texts and the social interactions during the campaign, which implies knowledge of the political context. Finally, we suggest an explanation by identifying the relationship between the social interactions and the broad social context, in particular the gendered dimension of politics. We conclude on its potential social effects.

‘Dynamic’ Aubry vs. ‘political’ Hollande (March-July 2011) In order to assess the lexical context for each NP with TERMOSTAT, we start by looking at the most frequent and the most specific words in the environment of the NP Hollande and the NP Aubry in the March-July 2011 corpus, i.e. in the 120 articles derived from the six newspapers. After listing the linguistic expressions (candidats de regroupement), TERMOSTAT gives the frequency of the given forms in the individual corpora and the specificity of associated items compared to an internal reference corpus5. First, we will consider the frequency criteria.

Frequency criteria Frequency criteria give an indication of the most common items used when covering a political campaign. The foremost lexical items found

4

Syntactic processes these NPs are associated the most with being the Subject of a sentence on the syntactic level, and with being an Agent on a semantic level (i.e. what thematic role they are exerting, role of patient, agent, etc. (cf. Siewierska 1991). With these abstracts we can observe whether NPs are constructed with more or less control of the verb. 5 The French Reference corpus (28,500 occurrences, ~560,000 inflected forms) is made of various articles published in 2002 by the daily newspaper Le Monde, http://termostat.ling.umontreal.ca/doc_termostat/doc_termostat.html.

Data Mining for Tracking Gender Bias

229

with the frequency criteria are very much the same for each candidate6. For instance, Table 1 gives the most frequent lexical items during the March-July period.

Items candidat primaire président secrétaire candidature député campagne parti sondage soutien projet maire

Hollande Frequency 290 248 171 170 153 150 121 108 97 83 82 72

Aubry Items candidat secrétaire projet primaire parti candidature socialiste sondage mai juin maire député

Frequency 300 234 212 204 161 157 109 94 90 85 83 82

Table 1: The most frequent items around the NPs Hollande and Aubry in March-July 2011 in the six newspapers If we remove the most common frequent units relating to the general domain of politics for both candidates, we obtain the exlusive lists given in table 2: Hollande

Aubry

capacité, chance, chemin, choix, compétition, débat, différence, droite, électeur, élu, expérience, face à face, favori, personnalité, ralliement, République, rien, rival, stratégie, strauss-khaniens, sympathisants, terrain, visite, voix

allié, appel, avenir, avril, bureau (national), canton, convention, emploi, entourage, envie, faveur, jeune, jeunesse, juin, leader, mai, mars, matin, mois, patronne, porteparole, programme, rassemblement, respect, ténor, texte, unité

Table 2: The most frequent items (alphabetical order) occurring around the NPs Hollande and Aubry, not common to both candidates

6

Thanks to Patrick Drouin for his advice on the matter of choosing the reliable threshold for frequency and specificity.

230

Chapter Twelve

We can note that the word favori (‘favourite’, i.e. preferred or most likely, frontrunner) is in the masculine7 and is only found in the Hollande corpus8. However, when we compare the most frequent words associated with the NPs Hollande and Aubry, it is Aubry who comes across as the ‘favourite’ and Hollande rather as the ‘challenger’. Firstly, the lexical field defined by the frequencies is generally more positive for Aubry than for Hollande: the general picture (topos – Anscombre 1995) that emerges from the data is one of a ‘politician-manager’. This is built up through a description of her as having leadership qualities with the words leader, porte-parole (‘spokesperson’), patronne (‘boss’), ténor (‘big name/heavyweight’ [i.e. in the party or in politics in general]), respect, faveur. This topos is also sustained by narrating what she does: she manages a team and an agenda (e.g. programme, ‘programme’; bureau, ‘office’; bureau national, ‘national office’; texte, ‘text’; employ, ‘job’). In contrast, Hollande is portrayed as a ‘politician-campaigner’ (e.g. voix, ‘votes’; élu, ‘elected’; électeur, ‘voter’; choix de stratégie, ‘choice of strategy’; visite, ‘visit’). Furthermore, Hollande is less often described as a leader (chef, ‘leader’; capacité, ‘ability’; favori). In particular, Hollande as a candidate seems embedded in a topos of ‘conflict’: he faces more challenges (rival; compétition; face ‘facing/opposing’; différence; strausskhaniens, ‘Dominique Strauss-Kahn supporter’) than Aubry does. Indeed Aubry seems to rally people (entourage; rassemblement, ‘gathering’; unité; allié, ‘ally’; convention) more than Hollande (sympathisants, ‘followers’; ralliement, ‘rallying’). These two portraits are the exact opposite of what we found with Royal and Sarkozy in 2007: Royal was described as a politician busy campaigning for the sake of winning and Sarkozy as a man on the move, busy managing social and economic situations. These results are also coherent with other research focusing on the speeches of the politicians. Indeed, Marchand & Ratinaux (2012: 692) list the most specific words as found in the socialist candidates’ speeches. For Aubry, out of 35 specific words, almost 40% are verbs, half of them denoting dynamism and actions such as battre (‘to beat’), augmenter (‘to raise’), arrêter (‘to stop’), défendre (‘to defend’), fermer (‘to shut down’); in contrast only 12% of the most specific words in Hollande speeches are verbs and they do not

7

For the feminine favorite (used in the Royal corpus in 2007), there is also a possible interpretation as ‘mistress’. 8 It is twice as frequent in the Hollande corpus (55 vs. 25) and does not appear in Aubry’s most frequent items.

Data Mining for Tracking Gender Bias

231

denote actions (falloir, ‘to have to’; avoir, ‘to have’; évoquer, ‘to refer to’). Knowledge of French politics lends support to this divisive portrait: Hollande was criticised from the outset for lacking charisma, not having the experience for a future president and trying to convince the party that he was up to the job. Indeed, Hollande had been the target of ridicule by influential people within his own party, such as Arnaud Montebourg (now Minister for industry and economic recovery) and Laurent Fabius (now Minister for foreign affairs). The latter gave him the nickname fraise des bois (literally ‘wild strawberry’) and the former flanby (a wobbly caramel pudding, sometimes misspelt flamby), because of his physique and his personality respectively. Both nicknames were used throughout the campaign in various blogs and forums by his adversaries until the very last day of the vote. Recourse to pejorative nicknames is a tactic known to be used more often against female than male politicians (Barré & Debras 1999; Bertini 2007). However, and surprisingly given these frequencies, the final two most frequent words, chance (‘luck’) and favori, only refer to Hollande, describing him as a possible winner. In other words, if frequency alone does not show a leader in the male candidate, maybe this feature will arise in analysing what we can consider to be the most ‘specific’ words.

Specificity criteria The data (Table 3) show that associated items (n-grams) listed according to specificity define much more personally the political profile of each NP under investigation. The specificities for each candidate give us an idea of the general ‘stereotypes’ to which each candidate could be reduced during the campaign or the general ‘qualities’ they are known for. Thus, we can go a step further than our initial analysis based on mere frequencies. The first topos we find is ‘socialist primaries’ for both candidates: primaire (socialiste), candidature (potentielle), (‘(potential) candidacy’), candidat (déclaré) (‘(declared) candidate’), parti socialiste, projet (‘programme or agenda’), investiture socialiste (‘socialist party investiture’). One unusual and shared expression is the adjective strauss-khanien, which defines a particular moment of the primaries and of the 2012 presidential campaign. This is the topos ‘Dominique Strauss-Khan’ (henceforth DSK) which refers to the main opponent of all of the socialist

232 Hollande N-grams primaire socialiste strauss-khanien(s) candidature president normal projet socialiste ex-premier secrétaire candidat normal projet socialiste investiture socialiste présidence normale sympathisants candidat déclaré humour corrézien député candidat de substitution hollandais position de favori biceps pro-dsk sauf Hollande présidentialité front anti-hollande experience ministérielle ralliement contrat de generation rocardiens stature présidentielle depot official opposant farouche justice fiscal referendum européen soutien président de conseil general challenger adversaire socialiste …

Chapter Twelve

Spec 155.97 113.69 67.11 47.89 44.49 52.38 47.89 44.49 44.03 43.77 41.72 39.32 38.73 38.73 34.11 32.54 29.17 28.54 28.13 28.13 28.13 28.13 27.81 25.82 24.33 24.33 24.33 24.33 24.33 24.04 24.04 23.29

Aubry N-grams primaire socialiste de Martine Aubry projet socialiste strauss-khanien(s) patronne emplois d’avenir conseil politique après DSK que Martine Aubry dit Martine Aubry rassembleuse présidentiable égalité réelle parti socialiste bureau national candidat declare primaire de designation soubassement intellectual convention nationale emplois jeunes délai de décence projet adopté texte de soutien coup de tonnerre jeunes socialistes partisans primaire d’octobre Martine Aubry fait réplique Aubry face à face Martine Aubry a themes économiques

Spec 122.00 69.52 64.95 60.24 49.52 45.02 35.93 35.08 35.08 35.08 35.08 34.89 34.06 33.92 32.79 31.99 28.93 28.93 26.64 25.15 25.03 24.74 23.31 22.75 22.40 21.68 21.25 21.25 21.25 21.25 21.25 21.25

22.68

coude à coude

21.25

22.65 21.79

image de rassembleuse candidature potentielle investiture socialiste candidature naturelle contrat de generation

21.25 21.25 20.40 17.39 17.33

Table 3. The most specific n-grams associated with the NPs Hollande and Aubry in March-July 2011 in the six newspapers

Data Mining for Tracking Gender Bias

233

candidates in the 2011 primary campaign. The DSK ‘saga’9 was the pivotal point of the elections and the March-July corpus covers the period before and after his arrest. The big question was the following: for whom would his (disappointed) followers be voting, Aubry or Hollande? The two previous topoi ‘politician-manager’ and ‘politician-campaigner’ bring out again the personal traits of the contenders, with an emphasis on their respective strategies: the dynamic Aubry (patronne) and the tactical Hollande (la normalité, ‘normality [i.e. an ordinary man]’). Martine Aubry is even more strikingly described as a person in action. Her dynamic personality and her determination are marked by NP+V clauses (as opposed to the more common NP+adj combinations): ce que Martine Aubry veut, dit, réplique, a, fait, etc. (‘what Martine Aubry wants, says, replies, has, does’). This dynamism is reinforced with terms and expressions referring to youth and the future and to a political agenda: emplois d’avenir, (‘jobs for the future’, i.e. as part of a scheme for getting young people to work), texte de soutien (‘supportive text’), emplois jeunes (‘youth employement’, i.e. as part of a scheme for getting young people to work), jeunes socialistes (‘young socialists’), thèmes économiques (‘economic themes’), égalité réelle (‘real equality’). Her personality is combative, and she seems ready to confront her adversary: patronne, coude à coude (‘neck to neck’, i.e. as in a close competition), face à face (‘one on one’, i.e. confrontation). No clauses such as Hollande dit/fait (‘Hollande says/does’) are to be found in the Hollande specificities. On the contrary, the collocations for which the NP is the central element are not positive and the topos of ‘conflict’ is more present: sauf Hollande (‘except for Hollande’), front anti-Hollande (‘anti-Hollande front’), challenger, opposant farouche (‘fierce opponent’), adversaire socialiste, pro-DSK. One unusual expression, candidat de substitution (‘replacement candidate’ [i.e. a second-rate candidate]) sums up rather well the negative connotations. This negativity is supported by other references to his former function as socialist party leader (ex-premier secrétaire) whereas for Aubry we find references to pending functions: premier secrétaire and maire (‘mayor’) of Lille. Even the positive expressions such as expérience ministérielle (‘ministerial experience’) and stature présidentielle (‘qualities of a president’) appear to be challenges rather than factual (or at any rate positive) comments. When we go back to the actual lines where these expressions appear we find, for 9

Dominique Strauss-Kahn, a former minister under various socialist prime ministers, and at that time at the head of the World Monetary Fund, was considered a contender for the French socialist party investiture until his fall from grace in the Sofitel sexual assault case.

234

Chapter Twelve

example, that Hollande’s presidential stature had been a bone of contention, i.e. he was thought to lack presidential charisma, necessary experience, personality, etc. What positive phrases there are are not present in sufficient numbers to counterbalance the effect of the largely negative ones. While Aubry comes across as ‘the boss’ (patronne), the paradigm of normalité is the most common (and no doubt the most ‘positive’) theme of the Hollande corpus: président normal, présidence normale, candidature normale. Interestingly, this theme of Hollande as ‘normal’ appears quite early in the primaries as a part of the Hollande strategy, as opposed to that of Nicolas Sarkozy. Indeed, Sarkozy was known for (and, as the opinion polls showed, was also disliked because of) his casual behaviour and language, which was interpreted by some as a lack of respect for the presidential function. However, Hollande’s ‘normality’ stance may also have targeted indirectly Aubry, serving to remind the socialist electorate that part of having a ‘normal presidency’ means electing a male candidate: electing a woman is not ‘normal’ or is dangerous, even (see CoulombGully 2012). Summing up, we can say that Hollande is definitely stigmatised in the journalistic discourse as an embattled politician trying to convince the world that he is credible. Aubry, on the other hand, comes across as determined, decisive and a team leader, i.e. she has all the typical male ‘qualities’. It is as though we have crossed stereotypes of male and female politicians: the female is active and described by her deeds and social functions; the male is described more by his personality and his flaws with the attribution of nicknames. All these Hollande personality traits were seized upon by various socialists and by Aubry herself (data March-July 2011, debate September 2011, etc.); Sarkozy, too, exploited these traits in the final debate on the 2nd of May 2012, reminding his rival of this criticism. Moreover, nicknames generally appear in connection with women politicians such as Merkel (Angie), Thatcher (Maggie, Iron Lady), etc.10 And yet, despite these linguistic portraits appearing to stigmatise the male politician, Hollande positioned himself all along as a winner, the likely opponent in the second round against Sarkozy. There is also, however, a potentially gendered aspect of his campaign through insistence on the ‘normality’ of his future style of presidency.

10

These reversed stereotypes were also witnessed during the three debates between Aubry and Hollande (Caillat 2012).

Data Mining for Tracking Gender Bias

235

Textual reading (KWIC) We examined the KWIC abstracts for Aubry and Hollande during the period of March-July 2011 in the reference corpus. Both candidates were at this time considered challengers since DSK was seen as the favourite until May. In this section we look at discriminatory discourse practices11 concerning the two main challengers in the March-July corpus, attempting to ascertain whether one of the two is more embedded in a negative discourse. This part of our study is a qualitative appraisal tracing the axiological values. We look at how mood, modality and lexical semantics as well as rhetoric devices such as irony and metaphor can help build up an impression of one or other of the candidates as being more active, more competent, and just better for the job. Thus it can be expected that a high number of negative terms (verbs such as ‘to ignore’ or ‘to refuse’; nouns such as ‘failure’ and ‘mistake’; adjectives such as ‘bad’ and ‘soft’, etc.) will convey a negative judgment of the person described. Negative contexts are more numerous for the NP Hollande than for Aubry. These contexts may be realised as negative adverbs or adverbs such as trop (‘too much’), un peu (‘a little’), etc. (cf. Michard & Ribéry 2008), as shown in examples 1 to 3: 1. François Hollande, lui, ne s’est pas encore manifesté ‘François Hollande has not yet shown himself’ 2. qui a un peu le même gabarit que l’ancien président ‘who is about the same size as the former president’ 3. Je pense que François (Hollande) part trop tôt. ‘I think François Hollande is starting too early’

Potentially negative items appear in the Aubry corpus to express support rather than criticism, as shown in examples 4 to 6: 4. (…) n’a pas caché son soutien à la gauche et à Martine Aubry pour la prochaine présidentielle ‘did not hide his support for the left nor for Martine Aubry for the next presidential election’ 5. responsable du rassemblement derrière Martine Aubry, il n’a jamais douté de sa volonté d'être candidate ‘responsible for the support behind Martine Aubry, he never doubted her willingness to be a candidate’ 11

We refer readers to our work (Baider & Jacquey 2010) on the description of agency and the methodology followed for its assessment.

236

Chapter Twelve 6. devant des militants déboussolés, Martine Aubry n’a eu de cesse de mettre en avant le programme ‘facing disoriented followers, Martine Aubry repeatedly stuck to her programme’

And when both candidates are compared, Aubry is the favourite, as we see in examples 7 and 8: 7. Contrairement à François Hollande, qui n’a jamais occupé de hautes fonctions ‘Unlike François Hollande who has never held a senior office’ 8. Elle peut faire l’unité mieux que François Hollande ‘She can unite people better than Hollande can’

In the environment of the NP Hollande, lexical content expresses disapproval and challenge, i.e. the opposite of what we find for Aubry which is associated with words of support, positive feedback and praise. Indeed, many of the occurrences of Hollande refer to actions being criticised or countered or quite simply too far removed from the harsh reality of politics (see examples 9-12). 9. La mobilisation de François Hollande agace les proches de Dominique Strauss-Kahn ‘Hollande’s all-out effort irritates the DSK camp’ 10. La candidature à ses yeux prématurée de François Hollande à la primaire socialiste ‘François Hollande’s candidacy in the socialist primaries is premature in his eyes’ 11. pour contrer la percée de François Hollande et pour dissuader d’autres éventuels candidats ‘to counter Hollande’s breakthrough and to deter other potential candidates’ 12. François Hollande est sur un petit nuage ‘François Hollande is on cloud nine’

By contrast, the lexical semantics of words surrounding the NP Aubry are positive about personality and actions (examples 13-21). Not only do the words chosen often express support but the newspaper Le Monde actually chooses Martine Aubry as its champion (example 13): 13. Le Monde, qui a désigné Martine Aubry ‘Le Monde which has chosen Martine Aubry’

Data Mining for Tracking Gender Bias

237

The NP Aubry is often followed by a verb of action (examples 14-17) or exulting power (example 18): 14. Martine Aubry a défendu le projet socialiste ‘Martine Aubry defended the socialist project’ 15. Martine Aubry dévoile ses priorités pour 2012 ‘Martine Aubry unveils her priorities for 2012’ 16. Il est reçu à la mairie de Lille par Martine Aubry qui a organisé en son honneur ‘He was received at the town hall by Martine Aubry who organised in his honour…’ 17. Martine Aubry a pour sa part réaffirmé ‘as for Martine Aubry, she reiterated…’ 18. Martine Aubry campe fermement dans son rôle de première secrétaire ‘Martine Aubry stands firm in her role as first secretary’

Instead of being on ‘cloud nine’, Martine Aubry is very much in the real world (example 19) and is trying to improve it (example 20) by her hard work (example 21) and careful planning (example 15): 19. politique de ‘care’ ou du ‘soin mutuel’ cher à Martine Aubry et la prise en compte du réel ‘politics of care and of mutual care, dear to Martine Aubry, and the taking into account of reality’ 20. Hier, Martine Aubry évoquait ainsi le retour des emplois jeunes ‘yesterday Martine Aubry mentioned the return of the youth employment scheme’ 21. Un travail de fond mené depuis deux ans par l’équipe Aubry va enfin payer ‘hard work over the last two years by the Aubry team will finally reap its rewards’

Whereas the modal verbs devoir and vouloir in the case of Aubry tend to be used neutrally, they are both used within a globally negative context for Hollande, as shown in examples 22 to 24: 22. Avant cela, François Hollande va devoir se distinguer des autres candidats socialistes ‘Before that, François Hollande will have to differentiate himself from the other socialist candidates’ 23. François Hollande veut surtout apparaître comme un prétendant crédible ‘François Hollande wants above all to appear as a credible candidate’ 24. François Hollande doit réfléchir à deux fois avant d'essayer ‘François Hollande has to think twice before trying’

238

Chapter Twelve

The negativity conveyed by the discourse in relation to the NP Hollande is striking: only 5% of the sentences can be labelled as positive evaluations, and 45% can definitely be considered as negative, as the occurrences listed in examples 22 to 24 showed. The repetition of words such as veut apparaître (‘wants to appear’), prétendant (‘a potential candidate’) or crédible portrays Hollande as wanting above all to put on an act and appear as a credible candidate. The opposite results are found in the discourse about Aubry, where more than 60% of the sentences are positively evaluated and only 5% can be seen as negative. She is portrayed as a leader, a politician who is hard-working, knows what she is doing, etc. It is surprising, then, that Hollande was given as the favourite in the opinion polls and went on to win the primaries. So, what happened in the month before the decisive vote? In the corpus focusing specifically on both candidates during the period from 15th September to 16th October, a shift in their linguistic portraits may provide some clues.

The second round: a month before the primaries Sept. - Oct. 2012 Frequencies and specificities Changes in the use of certain terms show an evolution in the ways Holande and Aubry are potrayed, with a decline of Aubry’s ‘lustre’. For instance, the clauses involving Aubry ‘doing’, ‘saying’, ‘replying’, etc. which had contributed to building an image of dynamism and action, disappear. Also, at a more personal level, her image and trump card of being a rassembleuse (‘a unifying force’) is nowhere to be seen (see Table 4). Aubry’s gender emerges as a salient category, with femme ‘woman’ and électorat féminin ‘female voters’ being present. We also find the noun police (Aubry had been the victim of a police investigation) and reference to her lawyer husband (mari avocat), which suggest intrusion into her private affairs12. The expression candidat(e) de substitution now appears in connection with Aubry whereas it was absent before the summer. These changes seem to embody a shift away from a ‘champion’ type of portrait, and more towards a traditional portrait of a female politician: the expression partie serrée (‘close contest’) sums up the tight race in which Aubry is now engaged. The opposite movement is true for Hollande. Expressions describing the struggle within his own party in the spring data 12

It can also be mentioned that numerous online forum postings contained very personal comments such as suspected alcoholism, her unattractiveness, etc.

Data Mining for Tracking Gender Bias

239

are no longer listed in the top frequencies. New expressions such as folie présidentielle (‘presidential folly’) and épreuve psychique (‘psychological test’) shift the focus from a struggle to a difficult, even unreasonable contest. Hollande’s personality still lacks the quality of a winner (réputation de sectaire, ‘known to be sectarian’) with his supposed arrogance being reflected in expressions such as forme de narcissisme (‘a kind of narcissism’) and the question of whether he will be able to ‘seduce’ the electorate (séduction nécessaire) is still not answered. Hollande N-grams candidat socialiste âge légal de départ (à la retraite) primaire socialiste règle d’or patriotisme industriel renégocier équipe de campagne candidat Hollande marge de manœuvre épreuve psychique projet socialiste stature présientielle traité européen folie présidentielle rancœur transition énergétique tractage (forme de) narcissisme capitaine de pédalo François Hollande bat impératif de rassemblement problème de pluralisme séduction nécessaire conseil politique réputation sectaire flou challenger redoutable

Aubry N-grams Spec 88.33 primaire socialiste 59.04 sympathisant de gauche

Spec 164.27 65.71 48.28 48.28 27.84 23.41 22.75 22.75

34.39 34.39 31.34 30.73 30.58 28.94 28.36 28.36 25.94 20.83 20.83 20.83 20.83

égalité salariale transparence démocratique langue régionale ex-première secrétaire sympathisant écologiste plan pluriannuel de rattrapage électorat féminin lassitude bougonne mi-despote mi-mère candidate de substitution Présidente stabilisation financière Avocet manque d’envie retour d’ascenseur Rosseries nuque raide pique envoyée

20.83 20.83 19.44 16.98 11.22 9.8

maire carré de fidèles voix étranglée discours de présidente femme ordinaire Challenger

15.76 13.11 13.11 13.11 13.11 6.61

55.68 48.71 44.12 38.18 35.94 34.39

22.75 22.75 22.75 22.75 22.75 21.40 16.07 16.07 16.07 16.07 16.07 16.07 16.07

Table 4. The most specific n-grams associated with the NPs Hollande and Aubry in September-October 2011 in the six newspapers

240

Chapter Twelve

Hollande’s gender is also indirectly referred to with the mention of football, a sport which was quite present later on during the SarkozyHollande duel, both men taking a special interest in this sport. This new feature subtly defines the political world as a typical male domain. The resurfacing of gender plays in Hollande’s favour: it helps him to fit into the presidential race, adding to the ‘normality’ paradigm and as well as adding virility to his persona.

Textual reading As far as the axiological value is concerned, even though criticism towards Hollande is ferocious, the polarity for Aubry has switched to a more balanced mix of positive and negative connotations. The female politician is still described according to her dynamism and is still perceived as a ‘politician manager’ (e.g. promet d’abroger, ‘promises to repeal’) and her strong personality is still prominent (e.g. sa pugnacité et son experience, ‘her pugnacity and her experience’; une femme qui a réussi à faire travailler le PS, ‘a woman who managed to get the socialist party working’; l’autre carte-maîtresse d’Aubry c’est son CV, ‘the other trump card for Aubry is her CV’) with a team supporting her (e.g. les fidèles de Martine Aubry l’encouragent, ‘Martine Auby’s followers encourage her’). However the ‘politician manager’ topos appears to have shifted to a position of ‘politician managing a crisis’. Aubry is described as being on the defensive, engaged in a struggle (e.g. intensifie le tir, ‘intensifies her attacks’; lâche ses coups, ‘delivers blows’; une attaque ravageuse de la part d’Aubry, ‘a devastating attack by Aubry’, etc.). In such a context, the expression Aubry est en dynamique (‘Aubry is on the move’) has a sense of Aubry lacking stability, and there is a hint of the cliché of the hysterical female. All her dynamism seems to be lost on an impossible objective, hence the ironic tone of most the previous expressions. Moreover, if her actions are now described as vain, her tactics (and therefore indirectly her ethics) come under greater scrutiny. Negativity refers to the accusations made by Aubry against Hollande’s character and depicts her as ruthless: le camp d’Aubry qui a fait de la médisance sa spécialité (‘the Aubry camp who practises gossiping as their specialty’) and Aubry is seen as trop heureuse de fustiger l’indécision et le flou (‘all too happy to criticise indecision and vagueness’). Interestingly, while everybody else six months earlier seemed to agree that Hollande’s personality was problematic, Aubry’s stance is now labelled as unfair behaviour when using the same argument; the mollesse (‘lack of courage’) of her opponent is also now

Data Mining for Tracking Gender Bias

241

only présumée (‘alleged’). Finally, the most damaging topos is her friendship with DSK which accounts for 15% of the negative occurrences. Hollande exploited this to damage her image and accused her of being a candidate de substitution pour DSK (‘a substitute candidate for DSK’). Nevertheless, Hollande also comes in for criticism and is still portrayed as a politician trying to convince the country he has what it needs: c’est un dangereux demagogue (‘he is a dangerous demagogue’), il est parti dans un délire (‘he has gone into a frenzy’), le look du tonton apaisant (‘the kind uncle look’), avec prudence ou fausse naïveté (‘with caution or false naivety’). However, even though he makes mistakes (il fait une erreur monumentale, ‘he makes a huge mistake’), he is portrayed as a winner – admittedly lacking in charisma, but still a possible winner (quite opposite from the situation three months earlier): Hollande c’est un vote utile (‘Hollande is a useful vote’), certes Hollande est arrivé en tête, mais (‘Hollande certainly came out as the winner but…’), un favori qui ne fait vraiment pas rêver (‘a favourite who is not really very inspiring’), un choix sans passion (‘a choice without passion’), etc. In spite of the negative discourse, Hollande went on to become the consensus of the left, when three months before Aubry was the rassembleuse: le vote Hollande c’est pour éviter la fragmentation (‘voting for Hollande will avoid divisions’), il est mieux à même de rassembler la gauche (‘he is better suited [than Aubry] to unite the left’). And as one extract from the September-October corpus sums up so well, Hollande a plus de chances de gagner contre Sarkozy mais Aubry le mériterait plus (‘Hollande is more likely to beat Sarkozy but Aubry deserves it more’). But why would Hollande be more likely to beat Sarkozy? Nothing in the data from March to May can explain why Hollande would be better suited for the job than Aubry. Nothing even in the September-October data can explain the lead he always had in the opinion polls. Neither his personality, nor his experience and his strategy were described as better than Aubry’s. Could the shadow of DSK have hung over Aubry’s head discrediting her indirectly? Or is it that the shadow of the unsuccessful candidate of 2007, Segolène Royal, reminded the socialist party of the danger of choosing a femal candidate? Did the opposition party, the UMP, who from the beginning set Hollande up as the candidat redouté (‘the feared candidate’) affect the vote in his favour? Or if Aubry’s profile and strategy coincided more with Sarkozy’s persona and tactics, could her rejection by the public be explained as a consequence of the massive rejection for the incumbent president? And why did the third favourite of the socialist

242

Chapter Twelve

party, Arnaud Montebourg, in the end decide to favour Hollande and not Aubry? Further in-depth research is needed to address these questions. In the present study we can only conclude that Aubry appeared to be more suited for the job in the discourse describing both candidates, but she still lost the investiture; and we can only add that Hollande, as an astute politician, saw his tactic of working on a personality opposing that of Sarkozy paying off in the end.

Conclusion: a question of status or gender? In conclusion, we can say that our close readings of newspaper discourse did not show any blatant gender stereotyping. In fact it was quite the opposite, since, before the DSK fiasco, we observed reverse stereotypes: Aubry was described as a dynamic and organised manager, Hollande as an embattled politician busy dealing with conflict. These same topoi were seen to correspond in 2007 to Nicolas Sarkozy and Ségolène Royal respectively. However, even with such a positive ethos, Aubry did not come out on top. In contrast, and despite a catastrophic portrayal in the media discourse, Hollande always led in the polls after the DSK fiasco. We also noted a shift in the linguistic portraits as seen in the SeptemberOctober data once it was clear that both were seen as potential frontrunners for representing the party against the incumbent president. This shift could again signal that once stakes are clearly higher, female politicians face negative discrimination. Both female contenders, Royal and Aubry, had a long political career behind them; Aubry’s game was the opposite of Royal’s, playing the card of toughness and being portrayed as competent. Her adversary, Hollande was also the opposite of Sarkozy, but in terms of having no charisma and no experience. But both female politicians lost, maybe, again, through fear of electing an atypical member of the French political class: whether from within a political party or in the public eye at large, being a woman, however powerful, could be the reason for both failures (Adler 1993; Lafrance 1997; Gerber 2009).

Bibliography Achin, C. & E. Dorlin 2008, “Nicolas Sarkozy ou la masculinité mascarade du président”, Raisons politiques 31, 19-45 Adler, L. 1993, Les femmes politiques, Paris: Seuil Anscombre, J.-C. 1995, “La théorie des topoï: sémantique ou rhétorique?”, Hermès 15, 185-198

Data Mining for Tracking Gender Bias

243

Baider, F. 2009, “Lexical change, discourse practices and the French press: plus ça change, plus c’est la même chose?”, In S. Tsiplakou, M. Karyolemou & P. Pavlou (eds.), Language Variation: European Perspectives II, Amsterdam: John Benjamins, 27-46 Baider F. & E. Jacquey 2010. “Substantive dis-embodiment, syntactic embedment”, In J. Holmes & M. Marra (eds.), Feminism, Femininity and Gendered Discourse, Newcastle: Cambridge Scholars, 145-167 Barré, V., S. Debras, N. Henry & M. Trancart 1999, Dites-le avec des femmes, le sexisme ordinaire des medias, Paris: CFD Bergvall, V. L. 1999, “Toward a comprehensive theory of language and gender”, Language in Society 28(2), 273-293 Bertini, M.-J. 2007, Femmes, le pouvoir impossible, Paris: Éditions Pauvert Bonnafous, S. & M. Coulomb-Gully 2007, “La journée internationale des femmes en France: entre marronnier et foulard islamique”, Sciences de la société 70, 81-95 Caillat, D. 2012. “Les recours au discours rapporté dans le débat présidentiel de l’entre-deux-tours de 2007: des usages sexués? ”, paper given at the Ci-dit conference, 14-16 juin 2012, University of Stockholm Cameron, D. 2003, “Gender issues in language change”, Annual Review of Applied Linguistics 23, 187-201 Coulomb-Gully, M. 2009, “Présidentiel 2007. Média, genre et politique”, Mots 90(1), 5-11 Coulomb-Gully, M. 2012, Présidente: le grand défi!, Paris: Payot Culioli, A. 1990, Pour une linguistique de l’énonciation. Opérations et representations (vol. 1.), Paris: Ophrys Dowty, D. 1991, “Thematic proto-roles and argument selection”, Language 67(2), 547-619 Drouin, P. 2003, “Term extraction using non-technical corpora as a point of leverage”, Terminology 9(1), 99-117 Eckert, P. & S. McConnell-Ginet 2003, Language and Gender, Cambridge: Cambridge University Press Fairclough, N. 1989, Language and Power, London: Longman Fairclough, N. 1995, Media Discourse, London: Edward Arnold Fracchiolla, B. 2011, “Politeness as a strategy of attack in a gendered political debate: the Royal-Sarkozy debate”, Journal of Pragmatics 43(10), 240-248 Gerber, G. L. 2009, “Status and the gender stereotyped personality traits: toward an integration”, Sex Roles 61: 297-316 Gingras, A.-M. 1997, “Les métaphores dans la langue politique”, Politique et sociétés 30: 159-171 Groussier, M.-L. 2000, “On Antoine Culioli’s theory of enunciative operations”, Lingua 110: 157-182 Grzesitchak M., E. Jacquey & M. Valette 2007, “Systèmes complexes et analyse textuelle: traits sémantiques et recherche d’isotopies”, paper given at the ARCo’07 conference, 28-30 November 2007, LORIA, Nancy Kahn, K. F. 1994, “The distorted mirror: press coverage of women candidates for statewide office”, Journal of Politics 56 (1), 154-173

244

Chapter Twelve

Kahn, K. F. 1996, The Political Consequences of being a Woman: How Stereotypes Influence the Conduct and Consequences of Political Campaigns, New York: Columbia University Press Kanter, R. M. 1977, Men and Women of the Corporation, New York: Basic Books Lafrance, M., H. Brownell & E. Hahn 1997, “Interpersonal verbs, gender and implicit causality”, Social Psychology Quarterly 60(2), 138-152 Lazar, M. M. (ed.) 2005, Feminist Critical Discourse Analysis: Studies in Gender, Power and Ideology, London: Palgrave Macmillan Lesage, R., P. Wilson, C. Bissonnette & P. Drouin 1993, “Enquête sur l’état d’utilisation des outils automatisés d’aide à la rédaction dans les organisations”, META 38(2), 367-389 Macaulay, M. & C. Brice 1997, “Don't touch my projectile: gender bias and stereotyping in syntactic examples”, Language 73(4), 798-825 Marchand, P. & P. Ratinaud 2012, “L’analyse de simitude appliquée aux corpus textuels: les primaires socialistes pour l’élection présidentielle française (septembre-octobre 2011) ”, In A. Dister, D. Longrée & G. Purnelle (eds.), JADT 2012 proceedings, Liège: University of Liège, 687‑ 699. http://lexicometrica.univ-paris3.fr/jadt/jadt2012/tocJADT2012.htm Michard, C. 1988, “Some socio-enunciative characteristics of scientific texts concerning the sexes, the nature of the right”, In G. Seidel (ed.), A Feminist Analysis of Order Patterns, Amsterdam: John Benjamins, 27-61 Michard, C. & C. Ribéry 2008 [1987], Sexisme et sciences humaines. Pratique linguistique du rapport de sexage, Lille: Presses universitaires du Septentrion Niven, D. & J. Zilber 2001, “Do women and men in congress cultivate different images? Evidence from congressional websites”, Political Communication 18(4), 395-405 Pauwels, A. 1998, Women Changing Language, London: Longman Penelope, J. 1990, Speaking Freely: Unlearning the Lies of the Fathers’ Tongues, New York: Pergamon Press Reutenauer, C., M. Valette & E. Jacquey 2010, “De l’annotation sémique globale à l’interprétation locale”, paper given at the Arco’09 conference, 9-11 November 2009, Rouen Roncarolo, F. 2000, “A woman at the Quirinal: thanks, but no thanks. The social construction of women’s political agenda in the 1999 Italian Presidential Election”, European Journal of Women’s Studies 7(1), 103-126 Siewierska, A. 1991, Functional Grammar, London: Routledge Sineau, M. 2010, La Force du nombre: femmes et démocratie présidentielle, Editions de l’Aube: Paris Sreberny-Mohammadi, A. & K. Ross 1996, “Women MP’s and the media representing the body politic”, Women in Politics 49(1), 103-117 Valette, M. 2010, “Propositions pour une lexicologie textuelle: les configurations du sens, In P. Blumenthal & S. Mejri (ed.), Zeitschrift für französische Sprache und Literatur 37, 171-188. Valette, M. & F. Rastier 2006, “Prévenir le racisme et la xénophobie – propositions de linguistes”, Les langues modernes 2, 68-77

Data Mining for Tracking Gender Bias

245

van Dijk, T. 2006, “Discourse and manipulation”, Discourse and Society 17, 359383 Widdowson, H. G. 2000, “On the limitations of linguistics applied”, Applied Linguistics 21(1), 3-25 Witt, L., K. M. Paget & G. Matthews 1994, Running as a Woman: Gender and Power in American Politics, New York: The Free Press

Lihat lebih banyak...

Comentários

Copyright © 2017 DADOSPDF Inc.