Democracy Indices and their biases
A deep dive into rankings published by Varieties of Democracy (V-Dem) institute
Puzzles generated by V-dem’s democracy index
The Sahel is one of the poorest regions in Africa comprising countries like Niger, Gambia and Mauritania. It is a coup-prone region, the most recent of which happened in 2023 in Niger. Niger has had 4 coups between 1974 and 2010. Yet, Niger is ranked at 109 in the V-Dem’s democracy index, one rank above India. Gambia has had multiple coups in its recent history. Gambia is ranked at 58 in the index. Mauritania had its first peaceful transfer of power about 5 years ago. It hovers slightly below India in V-dem’s rankings.
Let’s explore countries in Latin America. Mexico is ranked 10 places above India. It has had endemic political violence, with at least 17 mayoral candidates slain in the recent general elections. In Ecuador, during the August 2023 presidential elections, the opposition candidate was assassinated before the first round of voting could even take place. Ecuador is ranked at 66 in V-dem’s index.
Chile is going through a constitutional crisis, which is a good indicator that its democracy is facing problems. How is it ranked at 17 and classified as a liberal democracy? This is akin to an obese person being classified healthy. It casts serious aspersions on the methodology of the rankings. In Brazil, ex-President, Jair Bolsonaro was charged with attempting a coup, for refusing to accept the results of the 2022 election. Brazil ranks at 35 in the democracy index. What methodological flaw allows such countries to catapult to the upper quartile of democracies while a genuine large scale democracy is ranked in the bottom half?
Germany is trying to ban AfD, a right-wing extremist party. A quick view of their positions does reveal a few radical views, but nothing that other countries like India haven’t faced. The party managed to muster almost 5% of the vote, which is a decent level of support in the populace. Germany still comes in at number 11 in the democracy rankings.
Pakistan has faced every political turmoil possible. It has had coups, military dictatorships, puppet civilian governments controlled by the Army and a constitutional crisis. These are grave issues preventing grassroots democracy from taking root. It is shocking, on the verge of being hilarious, that Pakistan trails India only by 10 spots, sitting at 120.
What explains all these ranking discrepancies?
I could point out numerous other flaws with the ranking themselves. Why does the world’s largest democracy, arguably the most diverse, rank so low? India has had free and fair elections since becoming a republic in 1950. The two main political parties, BJP and Congress, regularly win and lose elections in various state elections. The current government suffered a major setback in the general elections earlier this year, giving the opposition a new lease of life. Out of the two house of Parliament, the current governing party doesn’t have a majority of its own in the Rajya Sabha.
There is raucous debate on news channels and on YouTube, with both pro and anti-government factions asserting themselves fiercely. Most major media houses are biased very sharply either in favor of the current dispensation or against it, with very few trying to remain neutral. There’s a case to be made that the same situation exists in most large democracies.
V-Dem classifies India as an electoral autocracy. What does that mean? In their own words
Over the years, India’s autocratization process has been well documented, including gradual but substantial deterioration of freedom of expression, compromising independence of the media, crackdowns on social media, harassments of journalists critical of the government, as well as attacks on civil society and intimidation of the opposition. The ruling anti-pluralist, Hindu-nationalist Bharatiya Janata Party (BJP) with Prime Minister Modi at the helm has for example used laws on sedition, defamation, and counterterrorism to silence critics. The BJP government undermined the constitution’s commitment to secularism by amending the Unlawful Activities (Prevention) Act (UAPA) in 2019. The Modi-led government also continues to suppress the freedom of religion rights. Intimidation of political opponents and people protesting government policies, as well as silencing of dissent in academia are now prevalent. India dropped down to electoral autocracy in 2018 and remains in this category by the end of 2023.
The summary that V-Dem puts out is highly subjective. India faces serious threats to territorial integrity, especially in border areas. Kashmir in the north and Arunachal Pradesh in the northeast are pressing territorial issues. Laws on sedition and counterterrorism are used liberally to control the situation. There are areas for improvement on this front, but India’s basic concerns must be contextualized in the rankings. No country tolerates secessionist tendencies.
The argument that the Modi government continues to suppress religious freedom doesn’t hold water. India has significant populations of all the major religions of the world. Muslims, the second largest majority in India, make up about 18% of the total population. The share of Muslims in the total population has increased since the subcontinent’s partition in 1947. Contrast this with Islamic States of Bangladesh and Pakistan where the population of Hindus has dropped precipitously in the decades following independence. The final nail in the coffin for this allegation comes from a Pew survey which states that 98% of Muslims feel free to practice their religion in the country. The same survey also reports that 95% of Muslims feel proud of their Indian culture and heritage. To me, that doesn’t sound like a minority facing religious restrictions.
Some states like Tamil Nadu and West Bengal have high levels of political violence. In contrast, states like Gujarat, Maharashtra, and Goa seldom experience election-related violence, whether during state or national polls. Tamil Nadu and West Bengal are ruled by non-BJP parties whereas the latter three are ruled by the BJP. So it is incorrect — bordering on malicious — for V-Dem to assert that the Modi-led government engages in political intimidation of its opponents when the exact opposite occurs.
What is V-Dem’s methodology?
To make sense of the rankings, I dug out their methodology from their website. I found the following summary on their website
V-Dem uses innovative methods to aggregate expert judgments and thereby produce estimates of important concepts. We use experts because many key features of democracy are not directly observable. For example, it is easy to observe whether or not a legislature has the legal right to investigate an executive. However, assessing the extent to which the legislature actually does so requires evaluation by experts with extensive conceptual and case knowledge.
V-Dem typically gathers data from five experts per country-year observation, using a pool of over 4,000 country experts who provide judgment on different concepts and cases. Experts hail from almost every country in the world, allowing us to leverage diverse opinions.
A quick glance at this raises obvious questions. For a country with as much diversity as India, are a handful of experts enough to pass a judgement? India has 6 national parties and 58 state parties and hundreds more parties that aren’t officially recognized by the Election Commission of India. There are very nuanced caste, religion and gender dynamics at play in every election. Each state is equivalent to a European country in terms of its area, population and language. India itself rarely has a single expert who are familiar with the politics of all of its regions, let alone all its states.
V-Dem has a set of core principles that they try to measure and then construct an Electoral Democracy Index (EDI) using those principles. They try to measure the following principles:
Electoral principle - whether elections are held periodically
Liberal principle - protects individual and minority rights against state repression.
Participatory principle1 - whether active participation occurs in all political processes.
Egalitarian principle - All groups enjoy equal de jure and de facto capabilities to participate.
Majoritarian principle - a majority must be capacitated to rule and implement their will in policy.
Consensual principle - a majority must not disregard political minorities
Out of these 6 principles, V-Dem doesn’t measure the majoritarian and the consensual principle because of the difficulty involved in measuring it. For the remaining principles, V-Dem has a set of indicators for each of the principles totaling about 470 in number.
Most of the work done in estimation of these principles and indicators is done by what V-Dem calls “Country Experts”. These experts are generally academics or policy people and are usually nationals or residents of the country in question. There are 5 experts recruited for each country. The experts are supposed to fill in each indicator and their responses are then averaged. As mentioned before, with humongous diversity, just 5 experts isn’t enough to evaluate India. At India’s scale, the subjectivity of the expert becomes the determining factor of the ranking. India has no shortage of experts both praising and criticizing it, making either choice likely to influence the outcome.
For instance, Arundhati Roy is an activist and author involved in various causes. She famously said that Maoists are just “Gandhians with guns”. She has professed animosity for the Indian state whichever party be in power. If she is recruited as a Country Expert, it is highly likely that India’s democracy index will suffer.
Some examples of indicators
There is an indicator for “Fraud allegations by Western monitors”. I find this indicator to be incredibly Western-centric. This assumes that an ideal democracy is the one being practiced in Western nations and that becomes the yardstick to evaluate other democracies. There have been many fraud allegations in Western elections, particularly in Canada and the US. Donald Trump and Pierre Poilievre have both leveled allegations of voter fraud. Why doesn’t that count in democracy rankings?
Another indicator exists for “Media Bias” and “Media Corrupt”. Again, incredibly subjective indicators highly dependent on expert’s politics. These would be unreliable in the best of times and especially so in a diverse country like India.
The “Ideology” and “Ideology Character” indicators ask country experts to characterize ideologies on a scale from Nationalist to Religious. This again assumes a Abrahamic definition of religious character of a nation. The ideology indicator is similarly subjective.
Impact of these indices
V-Dem doesn’t release the identity of Country Experts for fear of retribution and retaliation against them, which is fair enough. However, it must also be acknowledged that these rankings are subjective and different experts across the ideological spectrum would rate them differently.
There are other democracy indices similar to V-Dem’s. Two of the prominent ones are Freedom House based out of the US and the Economist’s Democracy Index. Both of them suffer from similar issues. While these are commendable academic exercises, being thorough in both depth and breadth of coverage, it is difficult to measure and compare subjective indicators across different countries, politics and cultures. At best, they could be a supplement to understand lacuna in a democracy.
Unfortunately, if it was a purely academic, this index could be ignored. But indices like these often feed into other indexes like World Bank’s World Governance Indicator. V-Dem’s indices are used to compute Regimes of the World index, Academic Freedom index, Women’s empowerment index and many more. S&P, Moody’s and Fitch’s use these indices to assign countries a letter grade to denote their credit ratings and credit worthiness. Getting a lower letter grade has a real world cost to India because it becomes expensive to avail of credit in the global market.
At this point, there needs to be more transparency in the computations of these indices. Chief Economic Advisor to the PMO, V Anantha Nageswaran has also emphasized the importance of transparency in these indices. Sanjeev Sanyal has also written a lot on the importance of managing narratives that may impact Sovereign Ratings. In a scathing indictment, he says that it “reminds him of money laundering”. There is hope that things may change, but no promising alternative appears on the horizon. India might be resigned to reactive narrative management.
In the latest V-Dem survey, Pakistan ranks about 15 places above India in the participatory component. Pakistan has issues with participation in its political processes, especially in the restive areas of Balochistan and the North West Frontier Province.