5 Supervised Scale Measurement II: Regression

Topics: Strategies for scale development with training data. Predictive modelling as measurement. Scales as linear functions of indicators. Estimating weights using training / “gold standard” data.

Required reading:

5.1 Seminar

This assignment follows the strategy used by the Economist Intelligence Unit’s “Where-to-be-born” index which aims to measure which countries tend to have the highest life satisfaction and therefore where you would want to be born in the “lottery of life”. The procedure followed by the Economist in 2006 was to start with data on the average life satisfaction in 130 countries, as measured by the Gallup World Poll. The Gallup prompt is:

Please imagine a ladder with steps numbered from zero at the bottom to 10 at the top. Suppose we say that the top of the ladder represents the best possible life for you, and the bottom of the ladder represents the worst possible life for you. On which step of the ladder would you say you personally feel you stand at this time, assuming that the higher the step the better you feel about your life, and the lower the step the worse you feel about it? Which step comes closest to the way you feel?

The Economist then used a multivariate regression to predict the country-level averages—the training data—as a function of a set of indicators:

“The life satisfaction scores for 2006 (on scale of 1 to 10) for 130 countries (from the Gallup Poll) are related in a multivariate regression to various factors. As many as 11 indicators are statistically significant. Together these indicators explain some 85% of the inter-country variation in life satisfaction scores. The values of the life satisfaction scores that are predicted by our indicators represent a country’s quality of life index. The coefficients in the estimated equation weight automatically the importance of the various factors. We can utilise the estimated equation for 2006 to calculate index values for year in the past and future, allowing for comparison over time as well across countries.

“The independent variables in the estimating equation for 2006 include: material wellbeing as measured by GDP per head (in $, at 2006 constant PPPS); life expectancy at birth; the quality of family life, based primarily on divorce rates; the state of political freedoms; job security (measured by the unemployment rate); climate (measured by two variables: the average deviation of minimum and maximum monthly temperatures from 14 degrees Celsius; and the number of months in the year with less than 30mm rainfall); personal physical security ratings (based primarily on recorded homicide rates and ratings for risk from crime and terrorism); quality of community life (based on membership in social organisations); governance (measured by ratings for corruption); gender equality (measured by the share of seats in parliament held by women).

“We find that GDP per head alone explains some two thirds of the inter-country variation in life satisfaction, and the estimated relationship is linear. Surveys show that, even in rich countries, people with higher incomes are more satisfied with life than those with lower incomes. However, over several decades there has been only a very modest upward trend in average life satisfaction scores in developed nations, whereas average income has grown substantially. The explanation is that there are factors associated with development that, in part, offset the positive impact. A concomitant breakdown of traditional institutions is manifested in the decline of religiosity and of trade unions; a marked rise in various social pathologies (crime, drug and alcohol addiction); a decline in political participation and of trust in public authority; and the erosion of the institutions of family and marriage.” https://www.economist.com/news/2012/11/21/the-lottery-of-life-methodology

The Economist has been continuing to use the relationships estimated from the 2006 data to update their estimates.

In this week’s exercise, we are going to “re-train” the model using more recent 2014-16 data from Gallup. It is entirely possible that life satisfaction has a different relationship to various possible indicators than it did in 2006. We are not going to examine the same indicators here, mostly because some of them are difficult to collect. Instead, we are going to use the four indicators from the 2018 Human Development Index: life expectancy at birth (LE), expected years of schooling (EYS), mean years of schooling (MYS), and GNI per capita (GNI). The variable LifeSat gives the average life satisfaction as measured by Gallup (2014-16).

lifesat_df <- read.csv("week-5-country-life-satisfaction.csv")
  1. Use pairs() to plot LifeSat and the four indicators LE, EYS, MYS and log(GNI) against one another, and cor() to check all the pairwise correlations.
  1. Fit a linear regression predicting LifeSat with the four indicators LE, EYS, MYS and log(GNI). Which of the country-level indicators most strongly predict country-level life satisfaction? Which of the country-level indicators do not predict country-level life satisfaction well, or have coefficients that seem to make less substantive sense?
  1. The expected years of schooling (EYS) and mean years of schooling are highly collinear (\(r\) = 0.82, and largely measure the same concept. Drop the one whose coefficient in the initial makes less substantive sense, and refit the linear regression.
  1. The data set includes HDI, which is calculated from these same indicators (look ahead to next week’s chapter of the textbook for details). How highly correlated are the fitted values from the regression that we just fit with HDI? How highly correlated is LifeSat with HDI? Explain the difference between these two correlations (this does not require looking up how HDI is calculated from the indicators).
  1. What are the largest positive and negative residuals in the training data? That is, which country has a Gallup measured life satisfaction that most exceeds what the fitted model predicts? Which country falls furthest below what the model predicts? Compare the indicator profiles, HDI and LifeSat for these two countries.
  1. Add Region to the linear regression model. Does it improve the fit of the model? Which regions have higher/lower levels of life satisfaction, holding constant the other indicators?
  1. Do you think it makes sense to include a variable like Region in this kind of measurement model? What are some arguments for or against?
  1. The Economist’s original measurement strategy was to calibrate the relationships between their indicators and country-level life satisfaction once (using data from 2006), and then use the resulting model with updated indicators in the future, without recalibrating on newer life satisfaction survey data. What is the key implicit assumption required for this to be a good measurement strategy?