Common mistake 1 with Difference-in-differences

Staggered adoption. When units are treated at different times, two-way fixed effects uses already-treated units as controls and can even get the SIGN wrong (Goodman-Bacon 2021). Use a modern estimator.

Common mistake 2 with Difference-in-differences

Not showing the pre-trends. Reviewers will assume you looked and did not like what you saw.

Common mistake 3 with Difference-in-differences

Too few clusters. With under ~40 treated units, cluster-robust standard errors are unreliable; use a wild bootstrap.

Common mistake 4 with Difference-in-differences

Choosing the control group after seeing which one gives parallel pre-trends. Searching over candidate controls until the pre-period lines up is fitting the assumption rather than testing it, and the resulting test has no meaning at its stated size.

EconStatsCraft › Difference-in-differences

Difference-in-differences

The workhorse of policy evaluation. Reports the 2×2 table, the two-way fixed-effects regression, an event study by period, and an explicit parallel-trends check.

What it needs

Outcome
Unit identifier - Region, firm, individual…
Time identifier
Treated group (0 / 1) - 1 for units that ever get treated.
Post-treatment period (0 / 1)
Controls - Optional.

The intuition

A before-after comparison confuses the policy with everything else that happened. A treated-versus-untreated comparison confuses the policy with pre-existing differences. Doing both and subtracting kills each problem with the other - provided the two groups would have moved in parallel absent the policy, which is the one thing you cannot test.

When to use it

A policy hits some units and not others at a known date, and you have data before and after for both.

Where the estimator comes from

With potential outcomes, \mathrm{ATT}=E[Y^1-Y^0\mid D=1]. The counterfactual E[Y^0_{\text{post}}\mid D=1] is unobserved; parallel trends assumes it equals E[Y_{\text{pre}}\mid D=1]+\bigl(E[Y_{\text{post}}\mid D=0]-E[Y_{\text{pre}}\mid D=0]\bigr). Substituting gives the difference of differences. In regression form, y_{it}=\alpha+\gamma\,\text{Treat}_i+\lambda\,\text{Post}_t+\delta(\text{Treat}_i\times\text{Post}_t)+u_{it}, and \delta=\mathrm{ATT}. Two-way fixed effects generalises this by absorbing arbitrary unit and period effects.

The model

y_{it}=\alpha+\gamma\,\text{Treat}_i+\lambda\,\text{Post}_t+\delta\,(\text{Treat}_i\times\text{Post}_t)+u_{it},\qquad \delta=\text{ATT}

What it assumes

PARALLEL TRENDS: absent the policy, the two groups would have moved in step. This is everything, and it is untestable - the pre-period event study is the closest you get.
No spillovers from treated to control units.
The composition of each group is stable over time.
Nothing else changed for one group and not the other at the same moment.

Statistical properties

Parallel trends is untestable by construction - it concerns a counterfactual. Flat pre-trends are supporting evidence, not proof.
Parallel trends is not invariant to transformation: parallel in levels is not parallel in logs.
Cluster at the level of treatment, not the observation - Bertrand, Duflo & Mullainathan (2004) showed ignoring serial correlation inflates t-statistics enormously.
With STAGGERED timing, two-way FE is a variance-weighted average of many 2×2 comparisons, some using already-treated units as controls, and the weights can be NEGATIVE - the estimate can carry the wrong sign even when every individual effect is positive.
Modern estimators (Callaway–Sant'Anna, Sun–Abraham, Borusyak et al.) fix this by only ever using not-yet-treated units as controls.

How to read the output

The interaction coefficient IS the treatment effect. The other two are baselines.
The event study is the credibility test: every pre-treatment coefficient should be near zero.
A pre-trend that is already diverging destroys the design - say so rather than hiding it.
Always cluster standard errors by the unit that was treated.

A worked reading

The 2×2 table does the whole job in four numbers: treated regions go from 23.7768 to 30.8688, a change of 7.0921; controls go from 21.8280 to 24.7728, a change of 2.9448; the difference of the differences is 4.1473. The regression returns exactly that, with a standard error of 0.1590 and a 95% interval of [3.8357, 4.4589], and the two-way fixed-effects specification reproduces it to four decimals - which is the point, since they are the same estimator written two ways. The event study is where the credibility sits rather than the coefficient: the pre-treatment effects are -0.1138, -0.0184 and 0.3305 against a 2017 baseline, none of them significant and all within a third of a unit of zero. That is what a defensible parallel-trends story looks like, and it is evidence for the assumption rather than proof of it.

Symptom → diagnosis → what to do

If you see this	It means	Do this
Pre-treatment event-study coefficients are significant	Parallel trends fails	Unit-specific trends, another control group, or abandon the design
Staggered treatment timing	TWFE can be badly biased	Callaway–Sant'Anna or stacked DiD
Few treated clusters	SEs unreliable	Wild cluster bootstrap, or randomisation inference
Group composition changes over time	Compositional confound	Use a balanced panel of the same units
Another policy hit the same group at the same date	Confounded	No statistical fix - argue institutionally
Effect appears BEFORE treatment	Anticipation, or mis-dated treatment	Check the timing; consider excluding the anticipation window

Common mistakes

Staggered adoption. When units are treated at different times, two-way fixed effects uses already-treated units as controls and can even get the SIGN wrong (Goodman-Bacon 2021). Use a modern estimator.
Not showing the pre-trends. Reviewers will assume you looked and did not like what you saw.
Too few clusters. With under ~40 treated units, cluster-robust standard errors are unreliable; use a wild bootstrap.
Choosing the control group after seeing which one gives parallel pre-trends. Searching over candidate controls until the pre-period lines up is fitting the assumption rather than testing it, and the resulting test has no meaning at its stated size.

Where to read this properly

Card & Krueger (1994) - the minimum-wage study that popularised DiD
Bertrand, Duflo & Mullainathan (2004) - "How Much Should We Trust DiD Estimates?"
Goodman-Bacon (2021) - the staggered-timing decomposition
Callaway & Sant'Anna (2021); Sun & Abraham (2021)
Roth, Sant'Anna, Bilinski & Poe (2023) - a practical review

All analyses

Data & variables

Each example is simulated from a known model, so you can check whether the estimator recovers the truth - and nothing here is anyone's private data.

Filter variables

Drag a variable into a slot on the right - or just click it.

Choose an example dataset above to begin.

Choose a transformation - it opens in the data editor, with a live preview.

Apply to

Multiply by

Columns ctrl-click for several

Group by ctrl-click for several

Within groups

Statistic

Bands

How many

Cut points

Keep rows where

Value

and

Fill with

Value

Also mark which rows were filled

Rules

Base period = 100

Which part

Percent at each tail

Threshold

Survey weight optional

Direction

Names column

Values column

Largest first

Delete them, not just count them

Read this column as

Tidy the text

New name optional

EconStatsCraft rigorous, plain-spoken econometrics

Econometrics that explains itself.

Regression, panel data, time series and the whole battery of tests. Every result is followed by a verdict you can defend.

Load a dataset to see it here.

Difference-in-differences

What it needs

The intuition

When to use it

Where the estimator comes from

The model

What it assumes

Statistical properties

How to read the output

A worked reading

Symptom → diagnosis → what to do

Common mistakes

Where to read this properly

See also

Related Causal

All analyses

Charts

Maps

Explore

Regression

Time series - one series

Time series - several series

Panel data

Multivariate statistics

Marketing research

Causal

Finance & risk

Actuarial

Tests & breaks

Machine learning

Econometrics that explains itself.