Common mistake 1 with Matching & weighting

Believing balance on observables implies balance on unobservables. It does not, and that is the whole problem.

Common mistake 2 with Matching & weighting

Including post-treatment variables in the propensity score.

Common mistake 3 with Matching & weighting

Reporting matching estimates without the balance table. It is unreviewable without it.

Common mistake 4 with Matching & weighting

Quoting the matching standard error as though the propensity score were known. It was estimated on the same data, and ignoring that understates the uncertainty - the module says so beside the number.

EconStatsCraft › Matching & weighting

Matching & weighting

Propensity-score matching, inverse probability weighting and regression adjustment side by side, with the covariate-balance table that decides whether any of it worked.

What it needs

Outcome
Treatment (0 / 1)
Covariates to match on

The intuition

If treatment was not random but you know WHY people got it, you can pair each treated unit with an untreated one that was equally likely to be treated. Rosenbaum and Rubin's insight was that you need not match on every covariate separately - matching on the single probability of treatment is enough, which turns an impossible high-dimensional problem into a tractable one-dimensional one.

When to use it

Observational data with a rich set of pre-treatment covariates and no instrument, discontinuity or natural experiment available.

Where the estimator comes from

Under unconfoundedness, (Y^0,Y^1)\perp D\mid X, and Rosenbaum & Rubin (1983) prove this implies (Y^0,Y^1)\perp D\mid e(X) where e(X)=P(D=1\mid X). So conditioning on the scalar score suffices. Matching estimates \widehat{\mathrm{ATT}}=\frac{1}{N_1}\sum_{i:D_i=1}\bigl(Y_i-Y_{j(i)}\bigr). IPW instead reweights: E\!\left[\frac{DY}{e(X)}\right]-E\!\left[\frac{(1-D)Y}{1-e(X)}\right]. Doubly robust estimators combine both and stay consistent if EITHER the score or the outcome model is right.

The model

e(\mathbf{x})=P(D=1\mid \mathbf{x}),\qquad \text{ATT}=E\!\left[y^{1}-y^{0}\mid D=1\right]

What it assumes

SELECTION ON OBSERVABLES: after conditioning on x, treatment is as good as random. Untestable and usually the weakest link.
Common support: treated and control units overlap in propensity score.
Only PRE-treatment variables in the score. Never include anything affected by treatment.
The propensity score model is correctly specified. Matching is only as good as the score, and a logit that omits an interaction which genuinely drives selection will balance the variables you included while leaving the sample unbalanced on the combination that mattered.
Treatment and control come from the same population and the same period. Matching a 2019 treated cohort to 2015 controls balances the covariates and leaves every difference the intervening years made.

Statistical properties

Everything rests on selection on observables - untestable, and usually the weakest link in the paper.
Balance on the variables you HAVE says nothing about the ones you do not.
Common support is essential: units with no comparable counterpart cannot contribute.
IPW is unstable when propensity scores approach 0 or 1, since weights explode. Trimming or stabilised weights help.
Matching standard errors that ignore the estimated score are too small (Abadie & Imbens 2016).
Never include post-treatment variables in the score - that is conditioning on a collider and it induces bias.

How to read the output

The balance table is the point. Standardised differences below 0.1 after matching means it worked.
The overlap plot: if the distributions barely intersect, the groups are not comparable.
Compare the naive difference with the matched estimate - the gap is the selection bias you removed.
If matching, IPW and regression adjustment disagree a lot, your result is driven by modelling choices.

A worked reading

The naive difference in means is 3.1085. Adjusting for the covariates cuts it to 1.4319 by regression, 1.5141 by inverse probability weighting and 1.2737 by nearest-neighbour matching within a caliper of 0.05 - against a true effect of 1.50 built into the sample. So more than half the raw gap was selection: better candidates were choosing into the programme. The worst standardised difference falls from 0.833 on prior score to 0.042 after matching, which is what makes the adjusted numbers worth reading at all. The three estimators disagree by about 0.24, which is the honest measure of how much the answer depends on the adjustment method rather than on the data - matching uses 863 pairs and discards the 75 units off common support, while regression uses all 1,500 and extrapolates over the same region.

Symptom → diagnosis → what to do

If you see this	It means	Do this
Standardised differences still above 0.1 after matching	Matching failed	Tighten the caliper, or accept the groups are not comparable
Little overlap in the score distributions	No common support	The effect is not identified for most of your sample
Some weights are enormous	Scores near 0 or 1	Trim, or use stabilised weights
Matching, IPW and regression disagree	Model-driven result	Report all three and say so
Estimate is sensitive to caliper choice	Fragile	Report a sensitivity table
A covariate is affected by treatment	Bad control	Remove it from the score

Common mistakes

Believing balance on observables implies balance on unobservables. It does not, and that is the whole problem.
Including post-treatment variables in the propensity score.
Reporting matching estimates without the balance table. It is unreviewable without it.
Quoting the matching standard error as though the propensity score were known. It was estimated on the same data, and ignoring that understates the uncertainty - the module says so beside the number.

Where to read this properly

Rosenbaum & Rubin (1983) - the propensity-score theorem
Dehejia & Wahba (2002)
Abadie & Imbens (2006, 2016)
Imbens & Rubin (2015), Causal Inference for Statistics, Social and Biomedical Sciences
King & Nielsen (2019) - "Why Propensity Scores Should Not Be Used for Matching"

All analyses

Causal

Finance & risk

Black–Scholes option pricing · Bond price, duration & convexity · Merton distance to default · EMI / home loan planner · CAPM / beta estimation · Multi-factor model · Portfolio optimization · Value at Risk & Expected Shortfall · Performance & drawdown · Time value of money & NPV/IRR · Equity valuation (DDM, FCF, residual income) · Cost of capital (WACC) · Binomial option pricing · Option strategies (payoff & profit) · Forwards, futures & swaps · Credit risk & Basel capital · Extreme value theory (tail risk)

Actuarial

Life table · Life annuity · Life insurance pricing · Chain ladder reserving · Buhlmann credibility · Survival analysis · Mortality graduation · Aggregate loss model · Ruin probability · Multi-state Markov model · Loss severity fitting

Tests & breaks

Monte Carlo experiment · F-test, Wald & Chow · Structural breaks & stability

Machine learning

Regression vs machine learning · Ridge / Lasso / Elastic Net · Random forest · Gradient boosting

Open the workspace →

Skip to results

Data & variables

Each example is simulated from a known model, so you can check whether the estimator recovers the truth - and nothing here is anyone's private data.

Filter variables

Drag a variable into a slot on the right - or just click it.

Choose an example dataset above to begin.

Choose a transformation - it opens in the data editor, with a live preview.

Apply to

Multiply by

Columns ctrl-click for several

Group by ctrl-click for several

Within groups

Statistic

Bands

How many

Cut points

Keep rows where

Value

and

Fill with

Value

Also mark which rows were filled

Rules

Base period = 100

Which part

Percent at each tail

Threshold

Survey weight optional

Direction

Names column

Values column

Largest first

Delete them, not just count them

Read this column as

Tidy the text

New name optional

EconStatsCraft rigorous, plain-spoken econometrics

Econometrics that explains itself.

Regression, panel data, time series and the whole battery of tests. Every result is followed by a verdict you can defend.

Load a dataset to see it here.

Matching & weighting

What it needs

The intuition

When to use it

Where the estimator comes from

The model

What it assumes

Statistical properties

How to read the output

A worked reading

Symptom → diagnosis → what to do

Common mistakes

Where to read this properly

See also

Related Causal

All analyses

Charts

Maps

Explore

Regression

Time series - one series

Time series - several series

Panel data

Multivariate statistics

Marketing research

Causal

Finance & risk

Actuarial

Tests & breaks

Machine learning

Econometrics that explains itself.