5 Panel Data and Difference-in-Differences

When we can observe and measure potentially confounding factors, we can recover causal effects by controlling for these factors. Often, however, confounders may be difficult to measure or impossible to observe. If this is the case, we need alternative strategies for estimating causal effects. One approach is to try to obtain data with a time dimension, where one group receives a treatment at a given point in time but the other group does not. Comparing the differences between pre- and post-treatment periods for these two groups allows us to control for unobserved omitted variables that are fixed over time. Under certain assumptions, this can produce valid estimates of causal effects.

Chapter 5 in MHE gives a very good treatment of the main empirical issues associated with difference-in-differences analysis, and with panel data more generally. The relevant chapter in Mastering ’Metrics is especially good for this week, so both are worth consulting.

The papers by Card (1990) and Card and Krueger (1994) are classics in the diff-in-diff literature, and give a very intuitive overview of the basics behind the method. More advanced applications can be found in the paper by Ladd and Lenz (2009), which also provides a useful demonstration of how difference-in-difference analyses can be combined with matching in order to strengthen the parallel trends assumption, and the recent paper by Dinas et al (2018), which we will replicate in part below.

An example of using fixed-effect regression to estimate a difference-in-differences type model can be found in the paper by Blumenau (2019).

For those of you who are feeling very committed, an important paper on statistical inference for difference-in-difference models is this one by Bertrand et. al. (2004). Be warned, however, that essentially no-one has ever enjoyed time spent reading a paper that is almost entirely about standard errors.

5.1 Seminar

This week we will be learning how to implement a variety of difference-in-differences estimators, using both linear regression and fixed-effects regressions. We will also spend time learning more about R’s plotting functions, as visually inspecting the data is one of the best ways of assessing the plausibility of the “parallel trends” assumption that is at the heart of all difference-in-differences analyses.

Data

We will use two datasets this week. The first is from a paper by Dinas et. al. (2018) which examines the relationship between refugee arrivals and support for the far right. The second is from a classic paper by Card and Krueger (1994) which examines the effects of changes in the minimum wage on employment. Both can be downloaded using the links at the top of the page.

5.1.1 Refugees and support for the far right – Dinas et. al. (2018)

The recent refugee crisis in Europe has conincided with a period of electoral politics in which right-wing extremist parties have performed well in many European countries. However, despite this aggregate level correlation, we have surprisingly little causal evidence on the link between influxes of refugees, and the attitudes and behaviour of native populations. What is the causal relationship between refugee crises and support for far-right political parties? Dinas et. al. (2018) examine evidence from the Greek case. Making use of the fact that some Greek islands (those close to the Turkish border) witnessed sudden and unexpected increases in the number of refugees during the summer of 2015, while other nearby Greek islands saw much more moderate inflows of refugees, the authors use a difference-in-differences analysis to assess whether treated municipalites were more supportive of the far-right Golden Dawn party in the September 2015 general election. We will examine the data from this paper, replicating the main parts of their difference-in-differences analysis.

The dinas_golden_dawn.Rdata file contains data on 96 Greek municipalities, and 4 elections (2012, 2013, 2015, and the treatment year 2016). The muni data.frame contained within that file includes the following variables:

  1. treatment – This is a binary variable which measures 1 if the observation is in the treatment group (a municipality that received many refugees) and the observation is in the post-treatment period (i.e. in 2016). Untreated units, and treatment units in the pre-treatment periods are coded as zero.
  2. ever_treated – This is a binary variable equal to TRUE in all periods for all treated municipalities, and equal to FALSE in all periods for all control municipalities.
  3. trarrprop – continuous (per capita number of refugees arriving in each municipality)
  4. gdvote – the outcome of interest. The Golden Dawn’s share of the vote. (Continuous)
  5. year – the year of the election. (Can take 4 values: 2012, 2013, 2015, and 2016)

Use the load function to load the downloaded data into R now.

load("data/dinas_golden_dawn.Rdata")
  1. Using only the observations from the post-treatment period (i.e. 2016), implement a regression which compares the Golden Dawn share of the vote for the treated and untreated municipalities. Does the coefficient on this regression represent the average treatment effect on the treated? If so, why? If not, why not?
post_treatment_data <- muni[muni$year == 2016,]
post_treat_mod <- lm(gdvote ~ treatment,data = post_treatment_data)

texreg::screenreg(post_treat_mod)
## 
## ======================
##              Model 1  
## ----------------------
## (Intercept)   5.66 ***
##              (0.25)   
## treatment     2.74 ***
##              (0.71)   
## ----------------------
## R^2           0.14    
## Adj. R^2      0.13    
## Num. obs.    96       
## ======================
## *** p < 0.001; ** p < 0.01; * p < 0.05

The treatment variable is a dummy measuring 1 for treated observations in the post-treatment period, and zero otherwise. This means that the regression estimated above is simply the difference in means (for support for the Golden Dawn) between the treatment group (municipalities that witnessed large inflows of refugees) and the control group (municipalities that did not receive large numbers of refugees). The difference in means is positive, and significant: treated municipalities on average were 2-3 percentage points more supportive of the Golden Dawn than non-treated municipalities in the post-treatment period. Notice that for this simple analysis, you would have produced exactly the same result by using the ever_treated variable rather than the treatment variable.

However, because the treatment was not assigned at random, we have little reason to believe that this difference in means would identify the causal effect of interest. The treated and control municipalities might very well have different potential outcomes, and so selection bias is – as always – a concern.

  1. Calculate the sample difference-in-differences between 2015 and 2016. For this question, you should calculate the relevant differences “manually”, in that you should use the mean function to construct the appropriate comparisons. What does this calculation imply about the average treatment effect on the treated?
# Calculate the difference in means between treatment and control in the POST-treatment period
post_difference <- mean(muni$gdvote[muni$ever_treated == T & muni$year == 2016]) - 
   mean(muni$gdvote[muni$ever_treated == F & muni$year == 2016])
post_difference
## [1] 2.744838
# Calculate the difference in means between treatment and control in the PRE-treatment period
pre_difference <- mean(muni$gdvote[muni$ever_treated == T & muni$year == 2015]) - 
   mean(muni$gdvote[muni$ever_treated == F & muni$year == 2015])
pre_difference
## [1] 0.6212737
# Calculate the difference-in-differences
diff_in_diff <- post_difference - pre_difference
diff_in_diff
## [1] 2.123564

Note that because the treatment variable only indicates differences in treatment status during the post-treatment period, here we need to use the ever_treated variable to define the difference in means in the pre-treatment period.

The difference in means in the post-treatment period (2.74) is larger than the difference in means for the pre-treatment period (0.62), implying that the average treatment effect on the treatment municipalities is positive. In simple terms, the difference-in-difference implies that the refugee crisis increased support for the Golden Dawn amongst treated municipalities by rougly 2 percentage points, on average.

  1. Use a linear regression with an appropriate interaction term to estimate the difference-in-differences. For this question, you should again focus only on the years 2015 and 2016. Note: To run the appropriate interaction model you will first need to convert the year variable into an appropriate dummy variable, where observations in the post-treatment period are coded as 1 and observations in the pre-treatment period are coded as 0. 
# Subset the data to observations in either 2015 or 2016
muni_1516 <- muni[muni$year >= 2015,]

# Construct a dummy variable for the post-treatment period. Note that the way it is
# constructed the variable in R means it is stored as a logical vector (of TRUE and FALSE 
# observations) rather than a numeric vector. R treats logical vectors as dummy variables, 
# with TRUE being equal to 1 and FALSE being equal to 0.
muni_1516$post_treatment <- muni_1516$year == 2016

# Calculate the difference-in-differences
interaction_mod <- lm(gdvote ~ ever_treated * post_treatment, data = muni_1516)
texreg::screenreg(interaction_mod)
## 
## ===============================================
##                                      Model 1   
## -----------------------------------------------
## (Intercept)                            4.39 ***
##                                       (0.24)   
## ever_treatedTRUE                       0.62    
##                                       (0.68)   
## post_treatmentTRUE                     1.27 ***
##                                       (0.34)   
## ever_treatedTRUE:post_treatmentTRUE    2.12 *  
##                                       (0.97)   
## -----------------------------------------------
## R^2                                    0.18    
## Adj. R^2                               0.16    
## Num. obs.                            192       
## ===============================================
## *** p < 0.001; ** p < 0.01; * p < 0.05

The regression analysis gives us, of course, the same answer as the difference in means that we calculated above. The coefficient on the interaction term between ever_treated and post_treatment is 2.12. Fortunately, regression also provides us with standard errors, and we can see that the interaction term is statistically significant.

  1. All difference-in-difference analyses rely on the “parallel trends” assumption. What does this assumption mean? What does it imply in this particular analysis?

The parallel trends assumption requires us to believe that, in the absence of treatment, the treated and untreated units would have followed similar changes in the dependent variable. Another way of stating the assumption is that selection bias between treatment and control units must be stable over time (i.e. there can be no time varying confounders).

In this particular example, this assumption suggests that, in the absense of the refugee crisis, treated and control municipalities would have experienced similar changes in the level of support for the Golden Dawn in the 2016 election.

  1. Assess the parallel trends assumption by plotting the evolution of the outcome variable for both the treatment and control observations over time. Are you convinced that the parallel trends assumption is reasonable in this application? Note: There are a number of ways to calculate the average outcome for treated and control units in each time period, and then to plot them on a graph. One solution is to use the aggregate() function which we used last week. Recall that the aggregate() function takes the arguments listed below. Here we need to calculate the mean of the gdvote variable by both year and whether the unit was ever_treated. It would also be possible to calculate each of the values manually and then store them in a data.frame for plotting, but this is likely to be very time consuming!
Argument Purpose
x The variable that you would like to aggregate.
by The variable or variables that you would like to use to group the aggregation by. Must be included within the list() function.
FUN The function that you would like to use in the aggregation (i.e. mean(), sum(), median(), etc) 
group_period_averages <- aggregate(x = muni$gdvote, 
                                 by = list(muni$year, muni$ever_treated), 
                                 FUN = mean)
names(group_period_averages) <- c("year", "treated", "gdvote")

There are many ways to plot the values that you have estimated. One way is to use the plot function. Although we have used this function before, there are many options that you can specify to customise the graphics that you produce. A few are included below, with comments next to the code to aid understanding. The code also makes use of the lines function. This is similar to plot, and the main arguments it takes are an x and y vector.

Another option is by using the gggplot2 package.

# base R 
plot(x = group_period_averages$year, # x specifies the variable plotted on the x-axis
   y  = group_period_averages$gdvote, # y specifies the variable plotted on the y-axis
   col = ifelse(group_period_averages$treated, "red", "blue"),  # col denotes the colour of the points 
   pch = 19, # pch determines the shape of the points in the plot
   xlab = "Year", # x-axis label
   ylab = "GD vote share", # y-axis label
   main = "Parallel trends?") # Plot title

lines(x = group_period_averages$year[group_period_averages$treated == T], 
    y = group_period_averages$gdvote[group_period_averages$treated == T], 
    col = "red")
lines(x = group_period_averages$year[group_period_averages$treated ==  F], 
    y = group_period_averages$gdvote[group_period_averages$treated == F], 
    col = "blue")

# ggplot
library(ggplot2)
library(ggthemes) 

ggplot(muni,aes(x=year,y=gdvote,colour=ever_treated)) +
   stat_summary(fun="mean",geom = "point") + 
   stat_summary(fun="mean",geom = "line") +
   ylab("% Vote Golden Dawn") +
   xlab("") +
   ggtitle("Parallel Trends?") +
   # If you want you can play around with the color scheme,
   # uncomment/comment out any of the ones below
   scale_color_economist("",labels = c("Treatment","Control")) +
   # scale_color_fivethirtyeight("",labels = c("Treatment","Control")) +
   # scale_color_excel("",labels = c("Treatment","Control")) +
   # scale_color_colorblind("",labels = c("Treatment","Control")) +
   theme_clean() +
   theme(legend.background  = element_blank())

The plot reveals that the parallel trends assumption seems very reasonable in this application. The vote share for the Golden Dawn evolves in parallel for both the treated and untreated municipalities throughout the three pre-treatment elections, and then diverges noticeably in the post-treatment period. This is encouraging, as it lends significant support to the crucial identifying assumption in the analysis.

  1. Use a fixed-effects regression to estimate the difference-in-differences. Remember that the fixed-effect estimator for the diff-in-diff model requires “two-way” fixed-effects, i.e. sets of dummy variables for a) units and b) time periods. Code Hint: In R, you do not need to construct such dummy variables manually. It is sufficient to use the as.factor function within the lm function to tell R to treat a certain variable as a set of dummies. (So, here, try as.factor(municipality) and as.factor(year)). You will also need to decide which of the two treatment variables (treatment or ever_treated) is appropriate for this analysis (if you are struggling, look back at the lecture notes!) 
fe_mod <- lm(gdvote ~ as.factor(municipality) + as.factor(year) + treatment,
                       data  = muni)

# Because we added a dummy variable for each municipality, there are many coefficients in this 
# model which we are not specifically interested in. Instead we are only interested in the 
# coefficient associated with 'treatment'. We can look at only that coefficient by selecting 
# based on rowname.
summary(fe_mod)$coefficients['treatment',]
##     Estimate   Std. Error      t value     Pr(>|t|) 
## 2.087002e+00 3.932824e-01 5.306624e+00 2.252636e-07

The key coefficient in this output is the one associated with the treatment indicator. Given the fixed-effects, this coefficient represents the difference-in-differences that is our focus. This model indicates an ATT of 2.087, which is very close to our estimates from the questions above (the small differences can be accounted for by the fact that we are now using all the pre-treatment periods, rather than just 2015).

  1. Using the same model that you implemented in question 6, swap the treatment variable for the trarrprop variable, which is a continuous treatment variable measuring the number of refugee arrivals per capita. What is the estimated average treatment effect on the treated using this variable?
fe_mod2 <- lm(gdvote ~ as.factor(municipality) + as.factor(year) + trarrprop,
                       data  = muni)

summary(fe_mod2)$coefficients['trarrprop',]
##     Estimate   Std. Error      t value     Pr(>|t|) 
## 6.061088e-01 1.324367e-01 4.576591e+00 7.082297e-06

Note that there is no difficulty in incorporating a continuous treatment variable into this analysis. The parallel trends assumption remains the same – that treated and control groups would have followed the same dynamics in the absense of the treatment, but here we can just interpret the treatment has having different intensities for difference units.

The coefficient on the trarrprop variable implies that the arrival of each additional refugee per capita increases the Golden Dawn’s vote share 0.6 percentage points on average.

Below is some code to put these all the models we’ve estimated together in one nice table.

library(stargazer)

# Re-run interaction model with interaction already calculated before to help with
# table formatting
muni_1516$treatment <- muni_1516$ever_treated*muni_1516$post_treatment
interaction_mod <- lm(gdvote ~ ever_treated + post_treatment + treatment,
                        data = muni_1516)

stargazer(post_treat_mod,
          interaction_mod, 
          fe_mod,
          fe_mod2,
          type = 'html',
          column.labels = c("Naive DIGM","Regression DiD","Two-way FE"),
          column.separate = c(1,1,2),
          keep = c("treatment","trarrprop"),
          omit = c(2),
          covariate.labels = c("Binary Treatment","Continuous Treatment"),
          keep.stat = c("adj.rsq","n"),
          dep.var.labels = "Golden Dawn Vote Share")
Dependent variable:
Golden Dawn Vote Share
Naive DIGM Regression DiD Two-way FE
(1) (2) (3) (4)
Binary Treatment 2.745*** 2.124** 2.087***
(0.711) (0.966) (0.393)
Continuous Treatment 0.606***
(0.132)
Observations 96 192 384 383
Adjusted R2 0.128 0.163 0.820 0.816
Note: p<0.1; p<0.05; p<0.01

5.1.2 Minimum wages and employment – Card and Krueger (1994)

On April 1, 1992, the minimum wage in New Jersey was raised from $4.25 to $5.05. In the neighboring state of Pennsylvania, however, the minimum wage remained constant at $4.25. David Card and Alan Krueger (1994) analyze the impact of the minimum wage increase on employment in the fast–food industry, since this is a sector which employs many low-wage workers.

The authors collected data on the number of employees in 331 fast–food restaurants in New Jersey and 79 in Pennsylvania. The survey was conducted in February 1992 (before the minimum wage was raised) and in November 1992 (after the minimum wage was raised). The table below shows the average number of employees per restaurant:

February 1992 November 1992
New Jersey 17.1 17.6
Pennsylvania 19.9 17.5
  1. Using only the figures given in the table above, explain three possible ways to estimate the causal effect of the minimum wage increase on employment. For each appraoch, discuss which assumptions have to be made and what could bias the result.

  1. Difference in means between the treatment and control groups in the post-treatment period. Assumption: no selection bias.

  2. Difference in means for the treatment group in the pre- and post-treatment periods. Assumption: no effect of time independent of treatment.

  3. Difference in differences. Assumption: Trend over time in the absence of the treatment is the same for treatment and control groups (parallel trends).

Replication exercise

The dataset m_wage.dta that you downloaded earlier includes the information necessary to replicate the Card and Krueger analysis. In contrast to the Dinas data, the dataset here is stored in a “wide” format, i.e. there is a single row for each unit (restaurant), and different columns for the outcomes and covariates in different years. The dataset includes the following variables (as well as some others which we will not use):

  1. nj – a dummy variable equal to 1 if the restaurant is located in New Jersey
  2. emptot – the total number of full-time employed people in the pre-treatment period
  3. emptot2 – the total number of full-time employed people in the post-treatment period
  4. wage_st – a variable measuring the average starting wage in the restaurant in the pre-treatment period
  5. wage_st2 – a variable measuring the average starting wage in the restaurant in the post-treatment period
  6. pmeal – a variable measuring the average price of a meal in the pre-treatment period
  7. pmeal2 – a variable measuring the average price of a meal in the post-treatment period
  8. co_owned – a dummy variable equal to 1 if the restaurant was co-owned
  9. bk – a dummy variable equal to 1 if the restaurant was a Burger King
  10. kfc – a dummy variable equal to 1 if the restaurant was a KFC
  11. wendys – a dummy variable equal to 1 if the restaurant was a Wendys

You will need to load the read.dta function in the foreign package (call library(foreign) before trying to call read.dta) to access this data.

library(foreign)
min_wage <- read.dta("data/m_wage.dta")
  1. Calculate the difference-in-difference estimate for the average wage in NJ and PA. Noting that the wage is not the outcome of interest in this case, what does this analysis suggest about the effectiveness of the minimum-wage policy? Note that there are some observations with missing data in this exercise (these are coded as NA in the data). You can calculate the mean of a vector with missing values by setting the na.rm argument to be equal to TRUE in the mean function.
# Pre-Treatment
pre_treatment_difference <- mean(min_wage$wage_st[min_wage$nj ==1],  na.rm = T) - 
   mean(min_wage$wage_st[min_wage$nj ==0],  na.rm = T)
pre_treatment_difference
## [1] -0.01799783
# Post-Treatment
post_treatment_difference <- mean(min_wage$wage_st2[min_wage$nj ==1],  na.rm = T) - 
   mean(min_wage$wage_st2[min_wage$nj ==0],  na.rm = T)
post_treatment_difference
## [1] 0.4633844
# Diff-in-diff
difference_in_difference <- post_treatment_difference - pre_treatment_difference
difference_in_difference
## [1] 0.4813823

The average starting wage rates are clearly very similar in New Jersey and Pennsylvania before the change in the minimum wage for New Jersey. The pre-treatment difference in means is about 2 cents, whereas the post-treatment difference in means is nearly 50 cents. This suggests that the minimum wage increase was successfully adopted in most NJ restaurants.

  1. Calculate the difference-in-differences estimator for the outcome of interest (the number of full-time employees). Under what conditions does this estimate identify the average treatment effect on the treated? What evidence do you have to support or refute these conditions here?
# Pre-Treatment
pre_treatment_difference <- mean(min_wage$emptot[min_wage$nj ==1],  na.rm = T) - 
   mean(min_wage$emptot[min_wage$nj ==0],  na.rm = T)
pre_treatment_difference
## [1] -2.891761
# Post-Treatment
post_treatment_difference <- mean(min_wage$emptot2[min_wage$nj ==1],  na.rm = T) -
   mean(min_wage$emptot2[min_wage$nj ==0],  na.rm = T)
post_treatment_difference
## [1] -0.1381549
# Diff-in-diff
difference_in_difference <- post_treatment_difference - pre_treatment_difference
difference_in_difference
## [1] 2.753606

The pre-treatment difference in means shows that New Jersey restaurants were initially significantly smaller in terms of full time employees in employment than Pennsylvanian restaurants, but the post-treatment period calculation suggests that this difference in FTE employment rates between NJ and PA stores had disappeared after the application of the treatment. Accordingly, the difference-in-differences (i.e. the difference in changes in FTE employment) is positive, and indicates that NJ stores had a relative increase in FTE employment compared to PA stores.

Of course, the crucial identifying assumption is that NJ and PA restaurants would have followed parallel employment trends in the absence of the minimum wage change. This assumption is very difficult to assess with the data we have here, as we do not have any information on employment outcomes for prior periods.

  1. Calculate the difference-in-differences estimator for the price of an average meal. Do restaurants that were subject to a wage increase raise their prices for fast–food?
pre_treatment_difference <- mean(min_wage$pmeal[min_wage$nj ==1],  na.rm = T) - 
   mean(min_wage$pmeal[min_wage$nj ==0],  na.rm = T)
pre_treatment_difference
## [1] 0.3086927
post_treatment_difference <- mean(min_wage$pmeal2[min_wage$nj ==1],  na.rm = T) - 
   mean(min_wage$pmeal2[min_wage$nj ==0],  na.rm = T)
post_treatment_difference
## [1] 0.3881344
difference_in_difference <- post_treatment_difference - 
   pre_treatment_difference

difference_in_difference
## [1] 0.0794417

The difference-in-differences estimate suggests that there is a very small increase (about 8 cents) in the average price of a meal after the introduction of the new minimum wage in NJ. It does not appear to be the case that the costs of the minimum wage increase were passed on to consumers.

  1. Convert the dataset from a “wide” format to a “long” format (i.e. where you have two observations for each restaurant, and an indicator for the time period in which the restaurant was observed). Estimate the difference-in-differences using linear regression. You should run two models: one which only includes the relevant variables to estimate the diff-in-diff, and one which additionally includes restaurant-level covariates which do not vary over time. Do your estimates of the treatment effect differ? Note: The easiest way to achieve the data conversion is to notice that you can simply “stack” one data.frame (with information from the pre-treatment period) on top of another data.frame (with information from the post-treatment period). So, first create two data.frames with the relevant variables. Second, bind these two data.frames together using the rbind() function (the data.frames must have the same column names before they are joined). Note that you will have to create the relevant treatment period indicator before binding the data.frames together. 
# Version 1
## Create two data.frames (one for pre-treatment and one for post-treatment period observations)
min_wage_feb <- min_wage[,c("nj","wage_st","emptot","kfc", "wendys","co_owned")]
min_wage_nov <- min_wage[,c("nj","wage_st2","emptot2","kfc", "wendys","co_owned")]

## Create a treatment period indicator
min_wage_feb$treatment_period <- 0
min_wage_nov$treatment_period <- 1

## Make sure the two data.frames have the same column names
colnames(min_wage_nov) <- colnames(min_wage_feb)

## Stack the data.frames on top of one another
min_wage_long <- rbind(min_wage_feb, min_wage_nov)

Another way of doing this would be with the dplyr package included in the tidyverse. The tidyverse includes a number of very useful packages (including ggplot2), that are used increasingly. Importantly, the syntax of the coding is a little different, as the packages rely a lot on what is called the pipe operator %>%, which you may have seen before. Here’s how you could answer q.4 (The usual disclaimers about the code apply: this is just ‘extra’, for those who are interested. If this is too much or too confusing for you right now, you can just ignore it :))

# Version 2
# install.packages("tidyverse")
library(tidyverse)

# First we gotta change some variable names and limit ourselves to the 
# variables of interest to make the pivoting easier afterwards
min_wage_long <- min_wage %>% 
   select(nj,kfc,wendys,co_owned,
          wage_0=wage_st,emptot_0=emptot,
          wage_1=wage_st2,emptot_1=emptot2) %>%
   # Now let's use pivot_longer(). It's a bit fiddly to figure out what to do and not. 
   # If using it in another context, be prepared for potentially a lot of annoying trial
   # and error.
   pivot_longer(cols = c(wage_0,emptot_0,wage_1,emptot_1),
                names_to = c(".value","treatment_period"),
                names_sep = "_")

You can use either one of the above to create min_wage_long to then estimate the ATT.

## Estimate the simple diff-in-diff
did <- lm(emptot ~ nj * treatment_period, min_wage_long)

## Estimate the covariate adjusted diff-in-diff
didcov <- lm(emptot ~ nj * treatment_period + kfc + wendys + co_owned, min_wage_long)

# Let's have a look at only the interaction to get the ATT's
stargazer(did, didcov,
          type='html',
          column.labels = c("Simple DiD","Covariate adj. DiD"),
          dep.var.labels = c("Total Number of FT Employed"),
          covariate.labels = c("ATT Estimate"),
          keep = c("nj:treatment_period1"),
          keep.stat = c("n","adj.rsq"),
          header = F)
Dependent variable:
Total Number of FT Employed
Simple DiD Covariate adj. DiD
(1) (2)
ATT Estimate 2.754 2.856*
(1.688) (1.526)
Observations 794 794
Adjusted R2 0.004 0.187
Note: p<0.1; p<0.05; p<0.01

There is very little difference between the regression adjusted estimate and the raw diff-in-diff estimate in terms of the size of the estimate. The standard error has decreased a little and the coefficient now about just is statistically significant at the 10% level.

5.2 Quiz

  1. How many potential outcomes does a unit have in a difference-in-differences framework?
  1. Two
  2. One per each time period
  3. Two per each time period
  4. Infinite
  1. What do we use the “parallel trends assumption” for?
  1. To randomise treatment assignment and timing of treatment assignment
  2. To substitute the unobservable untreated post-treatment potential outcome of the treatment group with potential outcomes we can observe
  3. To control for all time-varying characteristics that affect only the treatment group
  4. To observe the untreated post-treatment potential outcome of the treatment group
  1. What is the difference-in-differences?
  1. The difference between post-treatment and pre-treatment levels of Y in the treatment group minus the same quantity for the control group
  2. The difference between post-treatment and pre-treatment levels of Y for the treatment group
  3. The difference between post-treatment levels of Y of the treatment and control group minus the same quantity for the pre-treatment
  4. The difference between levels of Y of the treatment and the control group post-treatment
  1. Can we test the parallel trends assumption?
  1. No
  2. Yes
  3. Only in an experiment
  4. Only by including two-way fixed effects
  1. What do unit- and year-fixed effects hold constant?
  1. Characteristics that are specific to each unit and to each year respectively
  2. Characteristics that are specific to each unit and their time trends
  3. Characteristics that vary within-unit over time
  4. Characteristics that affect only the treatment group over time