1 Causal Inference and Potential Outcomes

This week we will introduce the topic of causal inference. We will outline a specific definition of causality using the potential outcomes framework, and will describe the fundamental problem of causal inference. We will highlight the persistent threat of selection bias in observational data and we will discuss differences between statistical inference and causal inference.

There are excellent introductions to the potential outcomes framework in Angrist and Pischke, 2014, (Introduction) and in Gerber and Green, 2012 (Chapter 1). The Paul Holland article on Statistics and Causal Inference provides an excellent history of the framework, and relates the conception of causality we focus on to long-standing philosophical discussions. If you are looking for inspiration, and a fun read, then the first half of David Freedman’s article on Statistical Models and Shoe Leather (p. 291-300) provides an interesting history of John Snow’s (no, not the guy from Game of Thrones) study of cholera in 19th century London. The second half of that article is also worth reading, as a fairly strong argument for why “statistical technique can seldom be an adequate substitute for good design, relevant data, and testing predictions against reality in a variety of settings.”

1.1 Seminar

For this week’s applied material we will review some key concepts of the “potential outcomes” framework and review some basics of using R. If you did not take PUBL0055 last term, or if are struggling to remember R from all the way back in December, then you should work through the exercises on the R refresher page before completing this assignment.

1.1.1 Potential Outcomes review

The following questions are designed to help you get familiar with the potential outcomes framework for causal inference that we discussed in the lecture.

  1. Explain the notation \(Y_{0i}\).
  1. Explain the notation \(Y_{1i}\).
  1. Contrast the meaning of \(Y_{0i}\) with the meaning of \(Y_i\).
  1. Can we observe both \(Y_{0i}\) and \(Y_{1i}\) for any individual unit at the same time?
  1. If \(D_i\) is a binary variable that gives the treatment status for subject \(i\) (1 if treated, 0 if control), what is the meaning of \(E[Y_{0i}|D_i = 1]\)?
  1. The table below contains the potential outcomes (\(Y_{1i}\) and \(Y_{0i}\)) and the treatment indicator (\(D_i\)) from a hypothetical experiment with 6 units. Complete the following calculations by hand.
    1. List the observed outcomes (\(Y_i\)) for the experiment based on the table above.
    2. Calculate the “true” average treatment effect (ATE) based on the potential outcomes.
    3. Calculate the “true” average treatment effect on the treated (ATT) based on the potential outcomes.
    4. Calculate the “estimated” average treatment effect based on the naive difference in group means for treatment and control conditions from the observed outcomes. Explain the difference between this estimate and the “true” average treatment effect.
Unit \(Y_{1i}\) \(Y_{0i}\) \(D_i\)
1 2 2 1
2 3 -1 1
3 -1 9 1
4 17 8 0
5 12 9 0
6 9 1 0

1.1.2 Islam and Authoritarianism

In a famous paper titled “Islam and Authoritarianism”, Steven Fish asks whether Muslim societies are less democratic.1 To find out, he runs a series of cross-sectional regressions of countries’ Freedom House scores (an indicator of the level of a country’s democracy) on characteristics of the countries, including whether they are predominantly Muslim.

The paper’s dataset is in the spreadsheet fishdata.csv, which you can download using the button at the top of the page. You should load the data using the read.csv() function, as follows:

This data contains the following variables (among others):

  • FHREVERS - Freedom House scores, a measure of democracy where higher values indicate that a country is more democratic and lower values indicate greater authoritarianism
  • MUSLIM - 1 if a country is predominantly Muslim, 0 otherwise
  • GDP90LGN - the country’s GDP in 1990
  • GRW7598P - the country’s average annual economic growth from 1975-98, in percent
  • BRITCOL - 1 if the country was a British colony, 0 otherwise
  • OPEC - 1 if the country is a member of the OPEC group of oil-exporting countries, 0 otherwise

We can look at the first 6 rows of this data using the head() function:

head(fish)
##   COUNTRY FHREVERS GDP90LGN ETHLING GRW7598P BRITCOL POSTCOM OPEC MUSLIM
## 1     Alb     4.10 2.925312    0.26     -0.8       0       1    0      1
## 2     Alg     2.15 3.214314    0.31      0.2       0       0    1      1
## 3     Arg     5.65 3.762078    0.21      0.6       0       0    0      0
## 4     Arm     3.95 3.187803    0.16     -6.6       0       1    0      0
## 5 Austria     7.00 4.435542    0.14      2.2       0       0    0      0
## 6  Austrl     7.00 4.255827    0.13      1.9       1       0    0      0
  1. Taking subsets and summarising variables
    1. How many countries are predominantly Muslim?
    2. What percentage of countries are predominantly Muslim?
    3. How many countries have GDP in 1990 of above 3.0?
    4. How many countries are both Muslim and a former British colony?
    5. How many countries have either average economic growth from 1975-98 of above 0.6% or GDP in 1990 of above 2.5?
    6. Create a new dataset consisting only of countries that are both Muslim and a member of OPEC

Code Hint: Use square brackets to denote subsets of a variable or dataset. You’ll also need the length() function.

  1. What is the difference in mean Freedom House score between Muslim and Non-Muslim countries? Calculate it by hand.
  1. Is the difference in means calculated above likely to be biased? If so, in which direction and why?
  1. Conduct a t-test for the difference in means calculated above using the t.test() function. Is the difference statistically significant?
  1. Conduct the t-test again, this time coding it by hand. Confirm that your answer is identical to the answer you calculated in the question above.
  1. Estimate a linear regression with FHREVERS as the dependent variable and MUSLIM as the independent variable. How do the results from your regression relate to the difference-in-means that you calculated in question 2?

  1. M. Steven Fish (2002). “Islam and Authoritarianism.” World Politics, 55 (1): 4-37↩︎