Common mistake 1 with Mortality graduation

Fitting Gompertz-Makeham across ages where it does not apply (below about 30, or above about 95) and reading the poor fit as a data problem.

Common mistake 2 with Mortality graduation

Running the chi-square test without exposure, or with exposure that is a headcount rather than a central exposed-to-risk.

Common mistake 3 with Mortality graduation

Over-graduating: forcing an extremely smooth curve that no longer reflects a real feature in the experience.

Common mistake 4 with Mortality graduation

Passing the tests and stopping. The tests detect departures they are designed to detect; a graduation can pass all three and still be visibly wrong at the oldest ages, where the data are thin and the chi-square contribution is small.

EconStatsCraft › Mortality graduation

Mortality graduation

Smooths a set of crude mortality rates by fitting a Gompertz-Makeham law, then checks the fit the way an actuary must: the chi-square test for overall level, the sign test for bias, and the runs test for one-sided drift, with a crude vs graduated plot.

What it needs

Age
Crude rate q_x

The intuition

Crude mortality rates computed age by age from finite data bounce around: with a thousand people aged 57 you might observe two deaths or five, and the rate jumps accordingly. But real mortality is smooth in age. Graduation replaces the crude rates with a smooth series that stays close to the data, trading a little fidelity for a great deal of plausibility - and the whole art is in choosing how much to trade.

When to use it

Producing a mortality table from a company's own experience, updating an industry table, or graduating any set of rates (sickness, lapse, retirement) estimated age by age from limited data.

Where the estimator comes from

Whittaker-Henderson minimises a penalised objective, \sum_x w_x(u_x-v_x)^2+\lambda\sum_x(\Delta^z v_x)^2, where u_x are crude rates, v_x the graduated ones, w_x weights (usually the exposure), \Delta^z the z-th difference and \lambda the smoothing parameter. The first term is FIT, the second is SMOOTHNESS, and \lambda sets the exchange rate between them. Differentiating gives the linear system (W+\lambda K'K)v=Wu, so the solution is a single matrix solve. With z=3 the penalty is zero for any quadratic, so a quadratic passes through untouched. The Gompertz alternative is parametric instead: fit \mu_x=Bc^x, or Makeham\'s \mu_x=A+Bc^x, by maximum likelihood.

The model

\hat{\mu}_x = A + B c^{\,x - x_0},\qquad \chi^2 = \sum_x \frac{(E_x\,q_x - E_x\,\hat{q}_x)^2}{E_x\,\hat{q}_x}

What it assumes

The underlying mortality really does follow a smooth law over the fitted age range - Gompertz-Makeham is a good description of adult mortality but not of the accident hump in early adulthood or the plateau at the oldest ages.
Deaths at each age are approximately Poisson given the central exposure, which is what the chi-square test relies on.
The crude rates are independent across ages (no duplicate lives contributing at several ages in a way that correlates the errors).
The exposure is measured on the same basis as the deaths - central exposed to risk against deaths in the same period, with the same definition of who is at risk. A mismatch between the numerator and denominator biases every rate in the same direction.

Statistical properties

Larger lambda gives a smoother series that follows the data less closely. There is no objectively correct value; it is chosen by inspection against the statistical tests below.
The difference order z sets what passes through unpenalised: z = 2 leaves straight lines alone, z = 3 leaves quadratics alone. Three is the usual choice for mortality.
Graduation must be tested, not just admired. The chi-squared test checks overall fit, the standardised deviations test their distribution, and the signs and runs tests check for systematic bias - a graduation can fit well on average while sitting above the data for twenty consecutive ages.
Over-graduation removes real features. If mortality genuinely has an accident hump around age 20, a heavy penalty will smooth it away and the graduated table will be wrong where it matters for that cohort.
A parametric law extrapolates beyond the data range; Whittaker-Henderson does not. For ages above the last observation a fitted Gompertz or a closure formula is required.
Weighting by exposure is essential. Unweighted graduation lets an age with fifty lives pull the curve as hard as one with fifty thousand.

How to read the output

The chi-square test checks overall fit: a small p-value means the graduated curve is too far from the data somewhere. It needs the exposure column.
The sign test checks for bias: roughly equal numbers of crude rates should sit above and below the graduated curve.
The runs test checks for smoothness of fit: too few runs of like sign means the curve is systematically off for long age stretches, even if the level is right.
A graduation that passes the level test but fails the runs test is over-smoothed; one that passes runs but fails chi-square is following the noise.

A worked reading

Fifty-six ages from 40 to 95 graduated by a Gompertz-Makeham law, fitting A = 2.40e-15, B = 0.000303 and c = 1.05594 - so mortality multiplies by about 5.6% a year of age, which is the compound growth that makes the curve look exponential on a log scale. The three tests that matter all pass, and they test different things. Chi-square is 44.74 on 53 degrees of freedom (p = 0.7831): the graduated rates are close enough to the crude ones overall. Signs come out 28 positive and 28 negative (p = 1.0000): the smoothing is not biased up or down. Runs gives 24 runs of like sign (p = 0.1775): the deviations are not clustered into long stretches where the curve sits above or below the data. Passing the first while failing the third is the classic signature of a graduation that is smooth but the wrong SHAPE, which is why all three are reported rather than a single goodness-of-fit number.

Symptom → diagnosis → what to do

If you see this	It means	Do this
Graduated rates hugging every wiggle	lambda too small	Raise it; the point is to remove sampling noise
A real feature smoothed away	lambda too large	Lower it, or model the feature explicitly
Signs test failing	Systematic bias: runs of positive or negative deviations	The shape is wrong, not just the smoothness
Chi-squared passing but runs failing	Fit is right on average and wrong in pattern	This is the failure mode the runs test exists to catch
Graduated rate falling with age in adulthood	Over-smoothing or a data error	Mortality should rise monotonically from about 30
Wild behaviour at the ends	Endpoint effects, common to all penalised smoothers	Extend the age range beyond what you need, then trim

Common mistakes

Fitting Gompertz-Makeham across ages where it does not apply (below about 30, or above about 95) and reading the poor fit as a data problem.
Running the chi-square test without exposure, or with exposure that is a headcount rather than a central exposed-to-risk.
Over-graduating: forcing an extremely smooth curve that no longer reflects a real feature in the experience.
Passing the tests and stopping. The tests detect departures they are designed to detect; a graduation can pass all three and still be visibly wrong at the oldest ages, where the data are thin and the chi-square contribution is small.

Where to read this properly

Whittaker (1923), "On a New Method of Graduation", Proceedings of the Edinburgh Mathematical Society 41.
London, Graduation: The Revision of Estimates (1985).
Forfar, McCutcheon and Wilkie (1988), "On Graduation by Mathematical Formula", JIA 115.
Continuous Mortality Investigation working papers (numbered series, ongoing), for current UK graduation practice.

Related Actuarial

Life table · Life annuity · Life insurance pricing · Chain ladder reserving · Buhlmann credibility · Survival analysis · Aggregate loss model · Ruin probability · Multi-state Markov model · Loss severity fitting

All analyses

Data & variables

Each example is simulated from a known model, so you can check whether the estimator recovers the truth - and nothing here is anyone's private data.

Filter variables

Drag a variable into a slot on the right - or just click it.

Choose an example dataset above to begin.

Choose a transformation - it opens in the data editor, with a live preview.

Apply to

Multiply by

Columns ctrl-click for several

Group by ctrl-click for several

Within groups

Statistic

Bands

How many

Cut points

Keep rows where

Value

and

Fill with

Value

Also mark which rows were filled

Rules

Base period = 100

Which part

Percent at each tail

Threshold

Survey weight optional

Direction

Names column

Values column

Largest first

Delete them, not just count them

Read this column as

Tidy the text

New name optional

EconStatsCraft rigorous, plain-spoken econometrics

Econometrics that explains itself.

Regression, panel data, time series and the whole battery of tests. Every result is followed by a verdict you can defend.

Load a dataset to see it here.

Mortality graduation

What it needs

The intuition

When to use it

Where the estimator comes from

The model

What it assumes

Statistical properties

How to read the output

A worked reading

Symptom → diagnosis → what to do

Common mistakes

Where to read this properly

See also

Related Actuarial

All analyses

Charts

Maps

Explore

Regression

Time series - one series

Time series - several series

Panel data

Multivariate statistics

Marketing research

Causal

Finance & risk

Actuarial

Tests & breaks

Machine learning

Econometrics that explains itself.