Beware of mashup indexes: how epidemic predictors got it all wrong

  

In October 2019, the Johns Hopkins University’s Bloomberg School of Public Policy, Nuclear Threat Initiative, and the Economist Intelligence Unit published, with significant publicity, the first World Health Preparedness Report and Global Health Security Index. It claimed to study the degree of readiness to confront epidemics country-by-country. In a report of 324 pages (in addition to a Website that allows to explore individual countries), the authors used six dimensions (or categories) to assess countries’ overall preparedness: prevention of the emergence of pathogens, early detection, rapid response, robustness of the health system, commitment to improving national health capacity, and overall (country’s) risk environment. The six categories themselves were built from 34 indicators, 85 subindicators and 140 questions. The authors then combined these six dimensions into an overall score, the index of Global Health Security (GHS). In this blog I shall be referring to that index.

The GHS index ranked 195 countries according to the number of points they have obtained from all the categories. The range was theoretically from 0 to 100, but the actual range went from 16.5 (the least prepared country; Equatorial Guinea) to 83.5 (the best prepared). The three top countries were the United States, UK, and the Netherlands.

As the “luck” would have it less than two months after the publication of the first global preparedness index, covid-19 struck the world with an unusual ferocity. So it is reasonable to ask how the experts’ judgments about various countries’ preparedness look compared to the actual outcomes in the fight against covid-19. For the latter, we use the number of covid-19 death per million inhabitants as of January 21, 2021. The data are collected from Worldometer. The death data are subject to many issues, from underestimation in many countries (as shown by the alternative statistic of excess deaths) to less frequent but possible overestimation. I will address these issues briefly below, and it will be indeed interesting to contrast GHS index with excess death data too.

  If GHS were to predict the outcomes of covid well, we would expect that countries with a high score would have lower rates of fatalities. Or alternatively, we could disregard the cardinal measurement, and look at the ranks where we would expect that higher ranked countries according to GHS would be also higher ranked in terms of how successful they are in fighting the virus (i.e., they would have relatively fewer fatalities). The second comparison is in some sense better because its requirements are less: it requires that GHS has broadly gotten correct the ranking of countries, not necessarily that it has successfully captured the absolute differences in outcomes.

Finally, note that GHS in principle already includes all information thought relevant for combating the pandemic. Thus adding to it factors that we believe might explain the outcome is inconsistent. Whatever experts believed was relevant was, by definition, included in the GHS index. Our objective in thus to test how successful were experts in choosing the relevant factors, in assigning them the correct weights, and coming up with an overall index.  

The answer is striking. The GHS index is not only unable to predict outcomes, that is, is not only orthogonal (unrelated) to the outcomes, but its rankings were often the inverse of the currently observed success rankings.  The two graphs below show the results. The graph on the left shows that GHS index is positively related to the rate of deaths—the very opposite of what we expect. The graph on the right shows that high ranked countries, like No. 1, the United States, No. 2, the UK, or No. 3, the Netherlands, are among the worst performers. Had the index got rankings correctly, we would have expected a 45 degree positively sloped line. On the contrary, we see that the US is ranked 145^th (out of 153 countries) according to its fatality rate: the difference between its predicted and actual rank is 144 positions! The UK, ranked by the preparedness index as the second best, is 149^th according to the actual outcomes.

For many rich countries, the gaps in ranks between the predicted and observed performance are enormous: to give a few examples, for France 124 positions, Italy 119 positions, Canada 99 positions, Germany 97 positions. On the other hand, many countries’ performances were much better than experts predicted: Vietnam was ranked No. 47, but in terms of performance it is No. 4; China 48 and 8; Cuba 95 and 19. There are thus many glaring discrepancies: Thailand and Sweden are ranked next to each other: the first recorded 1 death per million, the second 1078. Singapore and Argentina are also ranked together: Singapore had 5 deaths (per million), Argentina 1020. Several dozens similar comparisons can  easily be made.

 The exercise unmistakably shows that the predicted outcomes were far different (in some cases, the very opposite) of the actual outcomes. There are two possible defenses that the authors of the index can make.

First, it is very likely that the relative fatalities are mismeasured. But that argument is weakened by the fact that the differences in death rates between good and bad performers are enormous. They are often several orders of magnitudes different: the deaths per one million were (as of January 21, 2021), 1266 in the USA, 1 in Thailand, 3 in China, 16 in Cuba. However, mismeasured deaths are in the latter three countries, they cannot be underestimated by 1200+ times in Thailand, 400 times in China, or 80 times in Cuba. Moreover, for the index to make sense, equalizing China, Thailand and Cuba with the US, UK and the Netherlands is not enough: one would need to show that China, Thailand and Cuba did (as the index predicted) much worse—so the death mismeasurement requirements become truly astronomical. Thus, the exercise that might use excess death rates instead of reported deaths is almost certain to find the same lack of correlation between predicted and actual outcomes.

 The second defense is that the predictions made here referred in general to epidemics, while covid-19 is a very specific epidemic that tends to be much more fatal for the elderly or obese population. According to that argument, had the authors known the characteristics of covid-19, they would have produced a better GHS index. This is quite possible. But that belies the very idea underlying the index. If each epidemic is very idiosyncratic, then what is the purpose of having a general GHS index? Suppose that the next epidemic kills people with blue eyes. Since we do not know that such an epidemic will happen, what useful information can be gleaned from GHS index? We could then as reasonably make an altogether random ranking of countries—if each epidemic is entirely specific and its effects cannot be forecast.

   There is thus no escape from a sad conclusion that an index whose objective was to highlight strengths and weaknesses in the handling of potential epidemics has either entirely failed, or can be shown to have been useless. One can choose one or other of these two, equally damning, conclusions. But we should also make two additional points. First, study the (few) cases where the index successfully predicted the performance (they are in the SW corner of the second graph: Thailand, Australia, Singapore, Japan, Korea). Second, be wary of similar indexes that are produced for other variables like corruption, transparency in government, and the like. They too look “reasonable” until confronted by reality and may just reflect experts’ echo chamber thinking.    

On "Capitalism, Alone": On the occasion of Greek-language publication

 -If, as you say, capitalism is a system without an opponent, was Fukuyama right when he spoke of the “end of history”?

 Not really, because for Francis Fukuyama the end of history implied the domination of  liberal capitalism throughout the world. This is not what we observe. On the contrary, we note many forms of political organization, not just one. What I call “political capitalism” (as exemplified by China, Vietnam Singapore etc.). is one such non-liberal political form. But even with regard to capitalism, its current domination, which is both geographically and in terms of our system of values incontestable, should always be seen as a historical development, and never as a “terminus” of our  history.

 

-According to your interpretation, communism has fulfilled its role and is a system that belongs to the past and not to the future. Were the regimes of existing socialism communist? Only the Stalinists claim so. Today you will hardly find a Marxist who supports such a thing. On the contrary, many consider that the "existent" was the defamation of the socialist ideal.

I think that this is a very wrong view. Whether “the really existing socialism” was 100% compatible with Marx cannot be taken as a criterion whereby we judge whether it was “socialism” or not. There is no doubt that its essential characteristics, non-private ownership of the means of production and centralization of economic decisions, were fully in accord with traditional, including Marx’s, conception of socialism. Furthermore, we do not deny that today’s capitalism is “capitalism” even if some libertarians or even Friedmanites might not think so because of (say) too strong role of the state, existence of trade unions or high taxes. Such absolutely “pure” theoretical constructs, whether we speak of capitalism, socialism or feudalism have never existed. Likewise, we do not dispute that some societies are predominantly Christian even if they do not follow to the letter the tenets of the religion. Thus, “really existing socialism” was indeed socialism.

 

-You argue that the countries that have adopted the model of social democracy - one of the three of modern capitalism as you say - have succeeded in achieving levels of prosperity and political freedom unprecedented in human history. So can we say that this is still the solution to the problems facing societies today?

Yes, they have indeed done so in terms of economic prosperity, political freedom, and social mobility This is still a very good system—but I do argue that it was a system “constructed” for a world of homogeneous nation-state and of limited capital and labor flows. With much greater movements of capital and labor, the welfare state, which is the key part of social democratic capitalism, is under heightened pressure. Mobile  capital can leave if taxes are too high, and mobile (foreign) labor can move in. Loss of capital weakens the ability to fund the welfare state, while inflow of foreign labor reduces cultural homogeneity that was at the origin of the welfare state.

 

-What we see today is a turn of the “liberal capitalism”, which is becoming more and more authoritarian and moving closer to “political capitalism”, or closer to what some analysts call “capitalism with Chinese characteristics”. What do you think about it?

Yes, this is happening in some countries. But the crucial questions will be whether political capitalism would be able generate higher rates of economic growth than liberal capitalism. If so, it will eventually tend to dominate, either because other nations will also want to grow faster and might imitate some of technocratic features of political capitalism, or because the relative power of the countries of political capitalism will increase. This is, I think, in the background of the current US-China trade war: an ideological battle waged on the terrain of economic growth.

 

-How do you think that the present situation with the pandemic of corona virus  will affect democracy and social justice in general?

I think that it might lead countries toward reassessing (increasing) the role of the state in public services such as health care and education. First neoliberalism and then austerity policies have practically gutted many public services of people and instruments. Health care and education  began to be regarded like any other business activity. But they are not, because they have huge externalities: without heathy and educated population there is no economic activity and growth, and without fully healthy population even economic growth of a few is meaningless. This is why health and education cannot be managed the way we manage ordinary business: we cannot treat hospitals like hotels whose objective is to optimize the number of beds and patients. That approach is responsible for health providers becoming in many cases overwhelmed during this crisis and not able to save lives.

- Climate change is now perhaps the most important parameter in the evolution of human societies. Can economists in their theories ignore it or behave as if it does not even exist?

Of course, economists must address climate change. But they should do so using the traditional economic instruments of taxation and subsidies, taxing activities and products that contribute to emissions, and subsidizing their more ecologically-friendly alternatives. The arguments in favor of degrowth do not acknowledge these realities. Their proponents believe that the world can stop growing, but do not want to accept that this is possible only if either some 10-15% of the world population that lives in abject poverty stays poor forever, or if the rich world cuts its income by at least a third. Neither of these two options is desirable or indeed feasible, and this is why degrowth ideologues are wrong.

L’éducation idéologique

I believe that everyone can remember the turning points in their ideological and political evolution: these are moments when the entire world of ideas and preconceptions one has built over the years crumbles.  

I thought of three such events. I remember them with utmost clarity.

I.

I was in high school in Belgium during the Vietnam war. The whole atmosphere, from the school to newspapers, was suffused with anti-imperialism, and condemnation of the war. Being against the war not only seemed so normal that I had hard time imagining people who would be in favor of it, but, being a good young Marxistant, I assumed that the only people who could support killings of thousands of Vietnamese peasants must have been those who had an interest in the continuation of the carnage, namely capitalist financiers and the US ruling class.

The new, young and smart English teacher in our high school decided that we would learn English better if she would bring every week an issue of The Time or Newsweek magazine to read in class. It was a great success. She would then give the issue to a different student every week to take home and read. One time I was lucky to be given the magazine.  On the way back home, I opened it and saw a big photograph of Nixon visiting either Chicago or Detroit, and being greeted by hundreds of construction workers sitting on the scaffoldings, waving small American flags, cheering and supporting Nixon’s Vietnam policy. For a while I could not believe what I saw. I must have spent an hour or more (my walk back from school to home was about 40 minutes, and I was almost always walking alone), scrutinizing the picture: was it staged, were these people real workers or perhaps CIA operatives placed there to wave the flags? I thought of all these possibilities, very pleasing to my ideological beliefs, but still could not fully accept them. It did seem, as the article claimed, that Nixon’s reception was genuine and that the American working class was in favor of the war. I could not understand how that ideological “impossibility” could have happened. I was not able –a 17 year-old—to answer that question, but it remained like a big unresolved issue for a long time in my life. I was at the time of my first ideological doubts.

II.

In the 1980s the political situation in Yugoslavia was getting steadily worse, the recriminations between the republics more vociferous, the nationalist expressions of sentiment which in the past would be considered “hate speech” and drive the perpetrators to jail, were now commonly and openly voiced. Yet I believed that these were the bad residues of the tumultuous past and of the older generation, many of whom were supporters or collaborators of various Fascist factions. But surely, the young generation I thought –the youth being, by definition, progressive, anti-nationalist, anti-religious etc.—would be different.

With that view  in mind, I asked one day my older friend’s son, who was then in high school (probably of the same age as I was at the time of my first ideological epiphany with American workers), if his school classmates saw through these expressions of Serbian nationalist megalomania and stood for ethnic equality. So I asked him how other students thought of the Albanian issue, and how they believed it may be solved.

Oh, --he replied nonchalantly—we are all in favor of killing Albanians. And solving the problem once forever. 

III.  

A decade later I lived through the entire period of US hyperpower in Washington: the sole global power attacked, in short order, Panama, Serbia, Afghanistan, Iraq and Libya. I was aware of compete dissonance between reality and the way it was presented in the US media, having worked for the World Bank in transition countries in the 1990s, and especially in Russia. One of the reasons for my book “Income, inequality, and poverty during the transition to market economy” was precisely to leave a factual proof, to document in one place, what happened to poverty, inequality and destruction during the so-called “transition”. I am still very proud of that book, even if it is rarely cited.

I regarded the excesses of American hyperpower to be due to the military-industrial complex and Republican latter-day imperialists. I was not particularly interested in American domestic politics and thought that Democrats had, on the whole, little to do with the renascent imperialism. All of my friends were Democrats and they were sensible and nice people. In 2003, when the war on Iraq was launched I was at the Carnegie Endowment for International Peace, where –in consonance with its name—almost everybody was sincerely pained by what was about to transpire. I happened to be sitting in the office of a highly regarded and influential person who was brought almost to despair by Colin Powell’s fake UN testimony that paved the way to the aggression (and which was playing on TV live at the same time while I was sitting in that office).

So that was my view of liberals when a decade later I was invited to join an email list of relatively influential democratic media persons discussing current affairs. My first foray was based precisely on that assumption: that they were nice anti-imperialist people who cared about peace and the rest of the world. In no time  was I entirely disabused. With a high-level casuistry (since they were very intelligent and well-educated) they defended, and advocated, the most  destructive and murderous policies.

When today I heard of the title of Pankaj Mishra’s new book “Bland Fanatics” I was immediately gripped by its title. These were the people I was dealing with then! They were boring and lived in comfortable suburban homes. They penned most poisonous articles that would lead to the deaths of thousands while sipping Starbucks coffee and glancing from time to time at their daily “to-do” lists left in the morning by their spouse: “pick up the laundry”, “buy the spaghetti”, “call Jim to repair the AC”….Excitedly they would rush to wrap up their writings, reveling in more airstrikes, finishing the last paragraph perhaps too abruptly. For they had to pick up children from school. At four o’clock.

  

The Excellent Empire or the Prison of the Peoples? A review of Pieter Judson's “The Habsburg Empire: A New History”

 Writing a history of the Habsburg Empire from the Napoleonic wars to the dissolution of the Empire must be one of the most daunting tasks for a historian.  The variety of institutional and political arrangements, interacted with a bewildering multitude of social classes and nationalities that were in a state of permanent incipient conflict among themselves or with the Viennese center, makes such histories either too dull, as they become chronicles of events, or courting superficiality as they need to drop out a number of relevant developments to concentrate on a few they deem crucial.

Pieter Judson’s new “The Habsburg Empire: A New History” belongs to the latter  group. Judson decides to go for a revisionist tack where the second half of the 19^th century is argued to have been both economically and socially successful for the Empire. The Empire moreover is not considered too different (in its complexity) from other continental powers like Spain or France. Judson mentions, for example, several times that in 1863, one-fourth of the French population could not speak French. And then, the most contentious part, that the Empire might have found a way to create a sustainable federalist structure. I will concentrate, in this review, on the last point.

Judson needs to address two big issues here. First, why were nations that composed the Empire chronically unsatisfied, and why were institutional arrangements, reacting to this, in a state of a permanent flux? And second, why did the Empire ultimately dissolve, to the apparent delight of most, including the Austrians? For the latter, Judson blames the unreasonable harshness of the military dictatorship that took place at the onset of the Great War, as if that military dictatorship, which Judson acknowledges was much more hysterical and brutal  than in any other belligerent nation, could be considered in isolation from the nationality and social questions that plagued the Empire before the war.

However, the most important part of the book is whether the constitutional arrangements after 1867 in the Austrian part of the Empire (the Hungarian part was ruled much less inclusively by a narrow band of hidebound gentry) were sustainable. Judson’s main hypotheses are all made clear on two pages  (pp. 272-3) of a 500-page long book: “What made..the Austrian half of the dual monarchy unique... was not so much its ethnic make-up but rather the legal and administrative structures it developed to manage questions of linguistic and religious difference”. And, “[n]ationalist conflict was not an inevitable result of multilingual quality of Austrian and Hungarian societies but was the product of institutions…A political program that demands legal, social, and institutional rights for speakers of one of those languages would…encourage the people in a locality to see themselves and others largely in terms of language-based categories.”

It is those institutional arrangements that emphasized linguistic and ethnic diversity that, according to Judson, deepened ethnic cleavages and in the end broke the back of the Monarchy. Had the unifying and liberal bourgeoisie that tended to regard such differences with indifference and disdain prevailed, economic growth would have flattened out inequalities  between peasants and regional urban bourgeoisies, and then between different regions, from Bukovina and Galicia to Bohemia and Upper Austria. An economically vibrant precursor to the European Union would have been created, under a benevolent monarchical rule.

Why it did not happen was due to political agitation by nationalist politicians that found spreading nationalism a convenient way both to fight their own regional nobilities and acquire political power which, in the era of mass politics and increasing democratization in the Austrian lands (with the full male franchise introduced in 1907, that is, eleven years before Britain) was the way to power.

Judson thus sets the race between capitalist economic development on the one hand, and nationalist emancipation on the other. Had economic development been sufficiently strong to outpace nationalist demagogues the Austrian part of the monarchy might have found a durable solution for holding restless nationalities together.

This is not an unreasonable supposition. Economic development is often the glue that binds nations. “Material advantages will prove much stronger force for binding the peoples of the different crownlands” wrote a Viennese paper, quoted by Judson, in 1850. But two things must give us pause before we accept Judson’s institutionalist and economic arguments. First, the chicken-and-egg problem. The monarchy had to devise ever more complicated institutional fixes to preserve itself exactly because of national complaints. Thus institutional  responses cannot be blamed for creating the nationalities problem if they just reacted to something that already existed there. Second, it is economic growth and educational progress (notably in literacy) that created the nationalist intelligentsias and nationalist politicians whom Judson sees as responsible for sowing the seeds of dissension.

 Here we encounter the following problem: keeping the monarchy poor and uneducated could, in the short-run, reduce political problems, as Habsburgs surely knew how to negotiate and bargain with local nobilities. But in the medium term, it doomed monarchy to political irrelevance in Europe. To save itself from such a fate, the monarchy unleashed the forces of economic and educational progress, but that “created” nationalities and mass politics which then required new ethnic-based institutional framework. That framework eventually broke the country apart.

This was the key dilemma that the Habsburgs were unable to solve, and that Judson sidesteps. But the Habsburgs were not the only ones that failed here. The fact that the successor states of the monarchy, Czechoslovakia and Yugoslavia, and further East, Poland and the Soviet Union, showed themselves unable to solve the same problem should have made Judson think twice about his explanation.

For what is very obvious in the history of Central and Eastern Europe of the past two centuries is that all humanly possible political arrangements—centralized monarchy, decentralized monarchy, federalized monarchy, democratic state, dictatorial state, centralized republic, decentralized republic, feudal economy, capitalist economy, socialist economy—were all tried—and they all failed. Czechoslovakia and Yugoslavia collapsed at the first whiff of  international tensions and wars (in 1938-41 and then again 1989-92); Poland was structurally unstable through the inter-War years as Poles, Ukrainians and Jews vied for influence; the Soviet Union self-destructed in 1989-1992.     

Judson does in his Epilogue rightly point out that all of the countries that succeeded the Habsburg Monarchy replicated its problems, but he fails to draw the conclusion from the multitude of institutional solutions they tried, from 1918 to 1989, that it does not seem to have been a fixable problem.

It seems, with historical hindsight, that only the creation of more or less pure nation-states (which, with the exception of Bosnia, has been the product of the latest set of national revolutions) is consistent with cold peace in Eastern and Central  Europe. These are restless nations that, it seems, could find their peace only when left to their own devices, and freed from the distraction of inter-ethnic local politics.

 

 

Note: This is already a long enough review that I did not want to engage with a surprising omission of Habsburg foreign policy in Judson’s book. The numerous wars into which the Empire, more of less gratuitously careened (civil war in/with Hungary in 1848 that was “solved” only thanks to Russian intervention, then anti-Russian Crimean war mobilization in 1852-3, war with the Piedmont and France in 1859, support for Prussia against Denmark re. Schleswig-Holstein, then war with the self-same Prussia in 1866, annexation of Bosnia in 1908, and finally the World War in 1914) are treated as if they were so many falls of the meteorites and not revealing the deeply contradictory nature of the Empire. Moreover, wars' effects on domestic politics are left undiscussed: the 1867 agreement with Hungary was precipitated by the 1866 defeat against Prussia etc. Prussia is mentioned only four times in a book of more than 500 pages.