By Julian Dierkes
For some years, I have been collecting Mongolia’s score and rank on various global indices. I have also occasionally commented on some of these indices. Here, I want to focus on the Academic Freedom Index.
#Mongolia score declined slightly in #AcademicFreedomIndex drawing on @vdeminstitute.bsky.social data.
— Mongolia Focus (@mongoliafocus.bsky.social) 19. März 2025 um 09:57
What is the Academic Freedom Index?
From the AFI’s website:
The Academic Freedom Index (AFI) assesses de facto levels of academic freedom across the world based on five indicators: freedom to research and teach; freedom of academic exchange and dissemination; institutional autonomy; campus integrity; and freedom of academic and cultural expression. The AFI currently covers 179 countries and territories, and provides the most comprehensive dataset on the subject of academic freedom.
These five indicators are included in the V-Dem dataset. As is the case for V-Dem generally, this is an index that relies exclusively on scoring by experts. This is unlike other indices that rely primarily on information reported by national statistical offices like UNDP’s Human Development Index, or indices that include survey data like the Corruption Perception Index.
[Disclosure: I serve as an expert in several index projects, including V-Dem.]
Obviously, different methodologies bring different advantages and disadvantages with them. There are many global efforts focused on the UN to make statistical reporting comparable across nations, making indices based on such reporting most-easily comparable across countries. Of course, this assumes that national statistical offices operationalize data collection consistently and honestly. But, such statistical indices are clearly limited to topics that are meaningfully measurable by numbers.
Expert-based indices like the Academic Freedom Index, have the significant advantage that they can be conducted independently of the state, an aspect that may be most relevant to countries scoring low on various indices. Expert-based indices are inherently qualitative, i.e. they convert the assessment by an expert into a numerical score, which is a strength via independence and expertise, but a weakness in terms of validity and comparability. Few people can be considered an expert on multiple or even two countries, so that these expert-based survey attempt to assemble a list of experts around the world. There are many different ways in which organizers attempt to mitigate against different criteria or scales across experts by offering detailed instructions, including questions about confidence of specific judgements, or scoring experts themselves by offering them vignettes to score particular topics as a way to compare across experts. Some might also see a significant weakness in such expert-based indices in that they are virtually all based in OECD countries and, lo and behold, OECD countries generally rank high in these indices. Questions around comparability also make some of the expert indices inherently sticky or conservative in that they are looking for legislative changes or significant events to change the score for a country, particularly when the range of scores is limited.
Survey-based indices attempt to harness crowd wisdom by distributing the ranking of a given country across many more respondents than expert indices generally do. But, such survey indices are thus also susceptible to changes in the understanding of a given topic or in perceptions of governments. It is this later concern that also exists regarding expert surveys and will be a bit of a focus on the discussion below.
Why Global Indices
Accountability
Donors as well as voters might be looking for a way to assess the effectiveness of governments’ efforts on a particular topic. That is perhaps the dominant basis behind efforts related to the UN’s Sustainable Development Goals. Through a global process, such goals have been identified, say relating to girls’ education as an example. Governments can then be held accountable for their (lack of) success in reaching these goals.
Benchmarking
Governments may also want to compare themselves to other sets of countries to understand where they might want to focus their efforts because they are lagging countries they might consider good comparisons.
For different questions, one might want to compare to different countries. For example, the size of the population probably differentiates countries meaningfully when we are comparing social service provision. Mongolia and China may be neighbours, but their population numbers are on such a vastly different scale that few comparisons of social indicators make a lot of sense without a lot of qualifications. But, a relevant comparison group might be former state-socialist countries. For example, when looking at the development of (democratic) governance, comparing Mongolia to Poland may be of greater interest (shared starting point for democratization, roughly) because of their state-socialist history than South Korea even though democracy in South Korea has been operating for about as long as Mongolia. When looking at trade statistics, the Land-Locked Developing Countries might be the most relevant comparison.
Academic Comparisons
Any large-scale modelling of the behaviour of states will have to rely on some version of global indices to be included. For example, almost any comparative model is likely to include consideration of governance, some measure of democracy vs autocracy is generally seen as causally related to just about any state outcome. This is exactly where V-Dem enters the scene as perhaps the most academic of all the global indices. Perhaps not surprising, it thus includes measures of academic freedom.
How has Mongolia Done in the AFI?
Above, I have shown the graph produced on the Academic Freedom Index webpage, here’s the same data using the V-Dem graphing tools:
This graphs the five indicators for Mongolia for the democratic era, i.e. since 1990, that make up the Academic Freedom Index. After rising rapidly in the early 1990s with a new constitution and establishment of academic freedom, indicators remained stable until 2019. Campus integrity has fluctuated a bit, but gone from 2.01 in 2019 to current 1.69. Second highest is freedom of academic exchange and dissemination going from 1.47 in 2019 to current 0.86 Next, freedom to research and teach, 1.39 to 0.81 Second lowest is freedom of academic and cultural expression, 1.27 to 0.63 The lowest indicator is institutional autonomy, going from 1.31 to 0.53. Note that these changes are classified on the AFI website as “not significant”.
Compare some of these changes to other countries.
Compare those graphs to bigger trends around the world, for example OECD and Asia scores.
— Mongolia Focus (@mongoliafocus.bsky.social) 19. März 2025 um 11:46
Maybe one of the first things to note here is that scale matters. When you look at Mongolia’s score only (as in Bluesky post at the top), the change looks somewhat frightening or at least concerning. When you compare Mongolia’s score to other countries/regions, you may be frightened for the world (as long as you agree that academic freedom is an important element in democracy and good governance) and also concerned about the direction of Mongolia’s trend, but perhaps less so.
In a subsequent post, I will analyze how I understand Mongolia’s score.