Using topic modelling to analyse EU’s Rural Development policy

CONFERENCE PAPER

Using topic modelling to analyse EU’s Rural Development policy

René Brauer a and Mirek Dymitrow b a

Department of Philosophy, Linguistics, and Theory of Science, University of Gothenburg, P.O. Box 200, 405 30 Gothenburg, Sweden b

Department of Economy and Society, Unit for Human Geography, School of Business, Economics and Law, University of Gothenburg, P.O. Box 630, 405 30 Gothenburg, Sweden

Abstract Any social research at some point always touches upon issues dealing with the situatedness of the researcher. Policy analysis is no exception, and there have been many quantitative attempts to mitigate problems that arise from human biases. The general concern is that, for the most part, these methodological approaches remain fairly basic (e.g. word frequencies) in comparison to the semantic nuances a human reader would normally experience. In that light, the here presented approach explores the possibility of using topic modelling as a way to quantitatively assess policy without forgoing those finer nuances of human analysis. In this presentation, the material subject to topic modelling is EU’s Rural Development policy for 2007–2013 (RDP). It proclaims itself as the first European rural policy to in its conceptual framework cover aspects labelled as quality of life (QOL). In this paper, we treat this particular statement as a hypothesis, to which topic modelling is used to investigate if this really is the case. For any concept (not just QOL) to be incorporated into a policy and to retain its influence, it must be both mentioned and linked to other parts of that same policy, elsewise it could strike as merely a rhetorical strategy. Thereby, the hypothesis is that the more themes (topics) relate to a particular concept the more relative importance the policy allocates to that particular concept. In order to gauge this relative importance of QOL within RDP, we created a categorisation based upon: (1) the RDP’s own conceptual understanding of QOL, and (2) definitions provided by contemporary cutting-edge research dedicated to QOL. The analysis shows that only about 4 % of the topics found within the RDP relate to the issues of QOL. These particular topics only mention aspects of QOL without any explicit signs of implementation. This marginal position of QOL within RDP has been corroborated by other audits of the RDP using traditional qualitative techniques. Therefore, with further methodological development, this experimental application of topic modelling in policy analysis might represent one potential alternative to traditional qualitative methods.

Keywords: topic modelling, policy analysis, rural development, quantitative social science, digital humanities

Brauer, R. and Dymitrow, M. (2013). Using topic modelling to analyse EU’s Rural Development policy. Systematizing and digitalizing Nordic policy studies: Emergent perspectives within Swedish and Finnish research, Symposia, Aalto University, 27 November 2013, Helsinki, Finland.