Common mistake 1 with Staggered difference-in-differences

Running two-way fixed effects on staggered timing and reporting the single coefficient. Goodman-Bacon showed it is a weighted average that can carry NEGATIVE weights, and with growing effects it can come out the wrong sign entirely.

Common mistake 2 with Staggered difference-in-differences

Reading a flat pre-trend as proof of parallel trends. It is evidence, and a low-powered test fails to reject whether or not the assumption holds.

Common mistake 3 with Staggered difference-in-differences

Dropping the never-treated units to "keep the sample clean". They are the cleanest comparison group you have.

Common mistake 4 with Staggered difference-in-differences

Binning all post-treatment periods into one dummy when the cohorts have been treated for very different lengths of time. The average then depends on who has been treated longest, not on the effect.

Common mistake 5 with Staggered difference-in-differences

Treating a unit whose treatment switches off as never-treated, or as always-treated. Neither is right; look at why it switched.

EconStatsCraft Open the tool

EconStatsCraft › Staggered difference-in-differences

Staggered difference-in-differences

When units are treated at different dates, the usual two-way fixed-effects regression quietly uses ALREADY-TREATED units as controls, subtracting out their own effect. This estimator does not: it compares each cohort only with units not yet treated, and shows you the decomposition of what the naive regression was actually averaging.

What it needs

Outcome
Unit identifier - State, firm, school…
Time identifier
Treated now (0 / 1) - 1 in every period the unit is under treatment. The adoption date is read off it - you do not need a separate cohort column.

The intuition

For thirty years the standard way to handle a policy that rolled out in waves was to regress the outcome on a treatment dummy with unit and period fixed effects, and read off the coefficient. Goodman-Bacon asked what that coefficient actually IS, and the answer was uncomfortable: a weighted average of every two-by-two comparison available in the data - including ones in which a unit treated in 2012 serves as the CONTROL for a unit treated in 2016. In those comparisons the control group is already being treated, so its own effect is subtracted from the treated group's. If effects grow over time, the subtracted amount is growing too, and the estimate is pulled towards zero. With enough of these comparisons the weights on some of them turn negative and the sign of the estimate can flip while every unit gained from the policy.

When to use it

Any policy that rolled out in waves: states adopting a law in different years, firms upgrading on their own schedule, schools joining a programme over a decade. If every unit is treated on the same date, ordinary DiD is fine and this adds only the event study.

Where the estimator comes from

Callaway and Sant'Anna build up from the smallest well-defined object, the group-time effect ATT(g,t)=\mathbb{E}\left[Y_t(g)-Y_t(0)\mid G=g\right], identified under parallel trends and no anticipation as \mathbb{E}[Y_t-Y_{g-1}\mid G=g]-\mathbb{E}[Y_t-Y_{g-1}\mid \text{not yet treated}] - a clean two-period, two-group comparison with no already-treated unit anywhere in it. Aggregation is then a choice you make explicitly: \theta^{es}(e)=\sum_g P(G=g\mid G+e\le T)\,ATT(g,g+e) for the event study, or a cohort-size weighted average of all ATT(g,t) with t\ge g for a single number. Goodman-Bacon's result runs the other way, showing the TWFE coefficient equals \hat\beta^{DD}=\sum_{k\ne U}s_{kU}\hat\beta_{kU}^{2\times2}+\sum_{k\ne l}\left[s_{kl}^k\hat\beta_{kl}^{2\times2,k}+s_{kl}^l\hat\beta_{kl}^{2\times2,l}\right] with weights proportional to group sizes and to the variance of treatment within each pair - and the \hat\beta^{2\times2,l}_{kl} terms are the forbidden ones.

The model

ATT(g,t)=\mathbb{E}\!\left[Y_t-Y_{g-1}\mid G=g\right]-\mathbb{E}\!\left[Y_t-Y_{g-1}\mid \text{not yet treated at }t\right]

What it assumes

Parallel trends, conditional on not being treated yet. Treated and comparison units would have moved together in the absence of the policy - the pre-treatment path is the evidence, and the joint test reports it.
No anticipation: units do not change behaviour before the policy arrives. If they do, the period before treatment is contaminated and the whole baseline moves with it.
Treatment is absorbing. Once a unit is treated it stays treated; units that switch back are counted and flagged.
Nobody is treated in the first period, or that cohort has no before and cannot be used.
The panel is balanced enough that cohort averages compare the same units over time.

Statistical properties

ATT(g,t) needs no functional form. It is a difference of sample means between two groups over two periods, which is why it is robust to effects that differ by cohort and by duration - the thing TWFE cannot accommodate.
The "not yet treated" comparison group is legitimate and larger than the never-treated group. Units treated in 2018 are perfectly good controls for 2012 adopters in 2013, because in 2013 they are untreated. What is not legitimate is the reverse.
The Goodman-Bacon weights sum to one and the weighted average reproduces the TWFE estimate EXACTLY - not approximately. This app checks that identity to machine precision across equal and wildly unequal cohort sizes, because a decomposition that only nearly adds up has not decomposed anything.
Negative weights arise when a 2x2 comparison has very little treatment variance relative to the others. They are not a bug in the software producing them; they are a property of the estimand, and they are why the sign can flip.
Inference is bootstrapped over UNITS, because a unit appears in every period and its shocks persist. It is slightly optimistic in small panels: over 400 simulated panels of 24 units with a known effect, the nominal 95% interval covered the truth 92.5% of the time (standard error 1.1 points). That is close enough to be usable and far enough to be worth saying - with two dozen units there are only so many distinct resamples, and a normal interval of plus or minus 1.96 standard errors is a little too narrow. Widen it, or read the bootstrap percentiles directly, if the decision is close.
The event-study band is uniform (sup-t), not pointwise. A pointwise 95% band drawn over twelve coefficients excludes the truth somewhere with probability far above 5%, which is how a flat pre-trend gets called a violation.

How to read the output

The Callaway–Sant’Anna row is the average effect on the treated. The two-way fixed-effects row is what the naive regression returns, and the gap between them is the bias.
In the decomposition, "Later vs ALREADY-treated" is the forbidden comparison. Its weight tells you how much of the naive estimate rests on using treated units as controls.
In the event study, everything before period 0 is a TEST, not a result. Those coefficients should sit on zero.
The band is uniform, not pointwise: it covers the whole path at once. A pointwise band on twelve coefficients is wrong about one of them by construction.
A rising path after treatment means the effect grows - and a growing effect is exactly what makes two-way fixed effects understate it.

A worked reading

A programme reaching 24 states in four waves, with five never covered and an effect of 2.0 points on adoption growing 0.8 a year. Two-way fixed effects returns 2.72. The true average effect on the treated is 3.9, and the group-time estimator returns 3.86 with a bootstrap standard error of 0.68 - so the naive regression understates the effect by 30%. The decomposition says precisely where it went: 49% of the weight is on clean comparisons against never-treated states, 25% on earlier-versus-later cohorts, and 26% on later cohorts compared against states that were ALREADY treated - and that last group averages 1.27, less than a third of the true effect, because the control group was itself improving. The event study recovers the built-in path, rising from 1.6 on impact to 4.9 six years later, with the pre-treatment coefficients flat and a joint test at p = 0.24.

Symptom → diagnosis → what to do

If you see this	It means	Do this
TWFE far below the ATT	Effects grow and already-treated units are controls	Report Callaway-Sant'Anna; the decomposition shows the weight
Large weight on "later vs ALREADY-treated"	Many cohorts, long post periods	The naive estimate is mostly forbidden comparisons
Pre-treatment coefficients trending	Parallel trends is in doubt, or anticipation	Look for what changed before the policy; consider never-treated only
Very wide bands at long event times	Few cohorts are observed that long after treatment	Truncate the event window rather than reporting noise
A cohort treated in the first period	It has no pre-period	It is dropped - there is nothing to difference against
Effect vanishes with never-treated controls only	The not-yet-treated were driving it	Worth understanding before reporting either

Common mistakes

Running two-way fixed effects on staggered timing and reporting the single coefficient. Goodman-Bacon showed it is a weighted average that can carry NEGATIVE weights, and with growing effects it can come out the wrong sign entirely.
Reading a flat pre-trend as proof of parallel trends. It is evidence, and a low-powered test fails to reject whether or not the assumption holds.
Dropping the never-treated units to "keep the sample clean". They are the cleanest comparison group you have.
Binning all post-treatment periods into one dummy when the cohorts have been treated for very different lengths of time. The average then depends on who has been treated longest, not on the effect.
Treating a unit whose treatment switches off as never-treated, or as always-treated. Neither is right; look at why it switched.

Where to read this properly

Callaway and Sant'Anna (2021), "Difference-in-Differences with Multiple Time Periods", Journal of Econometrics 225(2).
Goodman-Bacon (2021), "Difference-in-Differences with Variation in Treatment Timing", Journal of Econometrics 225(2).
Sun and Abraham (2021), "Estimating Dynamic Treatment Effects in Event Studies with Heterogeneous Treatment Effects", Journal of Econometrics 225(2).
de Chaisemartin and D'Haultfoeuille (2020), "Two-Way Fixed Effects Estimators with Heterogeneous Treatment Effects", American Economic Review 110(9).
Roth, Sant'Anna, Bilinski and Poe (2023), "What's Trending in Difference-in-Differences?", Journal of Econometrics 235(2).

All analyses

Data & variables

Each example is simulated from a known model, so you can check whether the estimator recovers the truth - and nothing here is anyone's private data.

Filter variables

Drag a variable into a slot on the right - or just click it.

Choose an example dataset above to begin.

Choose a transformation - it opens in the data editor, with a live preview.

Apply to

Multiply by

Columns ctrl-click for several

Group by ctrl-click for several

Within groups

Statistic

Bands

How many

Cut points

Keep rows where

Value

and

Fill with

Value

Also mark which rows were filled

Rules

Base period = 100

Which part

Percent at each tail

Threshold

Survey weight optional

Direction

Names column

Values column

Largest first

Delete them, not just count them

Read this column as

Tidy the text

New name optional

EconStatsCraft rigorous, plain-spoken econometrics

Econometrics that explains itself.

Regression, panel data, time series and the whole battery of tests. Every result is followed by a verdict you can defend.

Load a dataset to see it here.

Staggered difference-in-differences

What it needs

The intuition

When to use it

Where the estimator comes from

The model

What it assumes

Statistical properties

How to read the output

A worked reading

Symptom → diagnosis → what to do

Common mistakes

Where to read this properly

See also

Related Causal

All analyses

Charts

Maps

Explore

Regression

Time series - one series

Time series - several series

Panel data

Multivariate statistics

Marketing research

Causal

Finance & risk

Actuarial

Tests & breaks

Machine learning

Econometrics that explains itself.