Common mistake 1 with Survival analysis

Dropping censored observations instead of using them - it throws away information and biases the curve downward.

Common mistake 2 with Survival analysis

Reading the flat right-hand tail of a KM curve as a real plateau when only a few lives remain at risk; the confidence band there is very wide.

Common mistake 3 with Survival analysis

Comparing groups with the log-rank test when the curves cross - the test can report no difference when the hazards differ a lot in opposite directions.

Common mistake 4 with Survival analysis

Quoting median survival when the curve never reaches 0.5. With heavy censoring the median is simply not estimable, and software that reports the last observed time instead is reporting the end of the study rather than a median.

EconStatsCraft › Survival analysis

Survival analysis

The non-parametric Kaplan-Meier survival curve and Nelson-Aalen cumulative hazard with right-censoring, the median survival time, a log-rank test across an optional group, and fitted Exponential and Weibull models with AIC.

What it needs

Duration observed
Event indicator (1 = event, 0 = censored)

The intuition

Some people have not had the event yet when the study ends, and throwing them away would be a disaster - they are the ones who lasted longest. Censoring is the whole problem survival analysis exists to solve. Kaplan-Meier handles it by working period by period: at each moment an event happens, ask what fraction of those still at risk survived it, and multiply those fractions together.

When to use it

Policy lapse and persistency analysis, mortality and morbidity studies, time to claim, equipment failure, or any duration data where following everyone to the event is impossible.

Where the estimator comes from

With distinct event times t_1<t_2<\dots, let d_j be the events at t_j and n_j the number still at risk just before. The Kaplan-Meier estimator is the product of conditional survivals, \hat{S}(t)=\prod_{t_j\le t}\left(1-\frac{d_j}{n_j}\right). A censored observation contributes to n_j up to its censoring time and then leaves the risk set without ever appearing in a d_j, which is exactly how it should count. Greenwood\'s formula gives the variance, \widehat{\mathrm{Var}}(\hat{S}(t))=\hat{S}(t)^2\sum_{t_j\le t}\frac{d_j}{n_j(n_j-d_j)}. The Cox model puts covariates on the hazard rather than the survival, h(t\mid x)=h_0(t)\exp(x'\beta), and estimates \beta from the partial likelihood without ever specifying h_0.

The model

\hat{S}(t) = \prod_{t_j \le t}\left(1 - \frac{d_j}{n_j}\right),\qquad \hat{H}(t) = \sum_{t_j \le t}\frac{d_j}{n_j}

What it assumes

Censoring is non-informative: a censored life is no more or less likely to have the event soon than one still under observation. Withdrawals that are related to risk break this.
The event time is measured on the same clock for everyone (policy duration, age, calendar time - pick one and be consistent).
For the log-rank test, the hazard ratio between groups is roughly constant over time; crossing curves make it lose power.
Entry into the study is not itself informative. If subjects are recruited after surviving some period - left truncation - the curve overstates survival unless the delayed entry is modelled.

Statistical properties

Kaplan-Meier is non-parametric and assumes nothing about the shape of the hazard. Its price is that it cannot extrapolate beyond the last observation.
It assumes censoring is INDEPENDENT of the event: people leave the study for reasons unrelated to their risk. If sicker patients drop out, the estimator is biased optimistic and no amount of data fixes it.
The curve is a step function that only drops at events, never at censoring times. Censoring shows as a tick and makes subsequent drops larger, because the risk set is smaller.
If the largest observation is censored the curve never reaches zero, and the mean survival time is not estimable. Report the median, which usually is.
The log-rank test compares curves and has most power when hazards are proportional. When curves cross it can return a p-value near one on groups that plainly differ.
Cox regression gives hazard ratios, not risk differences. A hazard ratio of 2 does not mean twice as many events, and proportional hazards is an assumption to be tested, not assumed.

How to read the output

S(t) is the probability of surviving beyond t. It is flat between events and steps down at each one; the size of the step is d_j / n_j.
The median survival time is the first t where the curve reaches 0.5. If the curve never gets that low, the median is "not reached".
A Weibull shape above 1 means the hazard rises with duration (wear-out, or increasing lapse pressure); below 1 means it falls (early shake-out). Lower AIC is the better parametric fit.
A small log-rank p-value says at least one group has systematically different survival - for lapse data, one distribution channel or product persists worse than another.

A worked reading

1,600 policies, 1,185 lapses and 415 still in force at the end - 25.9% censored, and every one of them carries information that dropping them would throw away. The Kaplan-Meier median survival time is 41.2. The estimate steps down only at observed event times (0.9496 at t = 7, 0.8817 at 12.3, 0.8143 at 16.9) and a censored observation leaves the risk set silently between steps, which is exactly how censoring is handled correctly rather than by guessing when those policies would have lapsed. The confidence band widens as the sample thins - ±0.011 at t = 7 against ±0.024 at t = 32.5 - because the tail of a survival curve rests on however few subjects remain. Two comparisons come free: a Weibull fit beats the exponential on AIC (11,714 against 11,869) with a shape of 1.396, so the hazard RISES with duration rather than staying flat; and the log-rank test separates the two channels decisively (chi-square 172.96, p < 0.0001), with broker business producing 613 lapses where 399 were expected.

Symptom → diagnosis → what to do

If you see this	It means	Do this
Curve not reaching zero	The longest observation was censored	Report the median, not the mean
A huge step late in the curve	Very few still at risk	The tail is unreliable; show the at-risk table
Curves crossing	Non-proportional hazards	The log-rank test loses power; use a landmark or time-varying analysis
Median never reached	Fewer than half had the event	Quote a different quantile, or restricted mean survival
Sicker subjects dropping out	Informative censoring	The independence assumption fails; the estimate is optimistic
Confidence band leaving [0,1]	The plain Greenwood interval on a raw scale	Use a log-log transformed interval

Common mistakes

Dropping censored observations instead of using them - it throws away information and biases the curve downward.
Reading the flat right-hand tail of a KM curve as a real plateau when only a few lives remain at risk; the confidence band there is very wide.
Comparing groups with the log-rank test when the curves cross - the test can report no difference when the hazards differ a lot in opposite directions.
Quoting median survival when the curve never reaches 0.5. With heavy censoring the median is simply not estimable, and software that reports the last observed time instead is reporting the end of the study rather than a median.

Where to read this properly

Kaplan and Meier (1958), "Nonparametric Estimation from Incomplete Observations", JASA 53(282).
Cox (1972), "Regression Models and Life-Tables", JRSS-B 34(2).
Klein and Moeschberger, Survival Analysis (2003), ch. 4 and 8.
Therneau and Grambsch, Modeling Survival Data (2000), for testing proportional hazards.

Related Actuarial

Life table · Life annuity · Life insurance pricing · Chain ladder reserving · Buhlmann credibility · Mortality graduation · Aggregate loss model · Ruin probability · Multi-state Markov model · Loss severity fitting

All analyses

Data & variables

Each example is simulated from a known model, so you can check whether the estimator recovers the truth - and nothing here is anyone's private data.

Filter variables

Drag a variable into a slot on the right - or just click it.

Choose an example dataset above to begin.

Choose a transformation - it opens in the data editor, with a live preview.

Apply to

Multiply by

Columns ctrl-click for several

Group by ctrl-click for several

Within groups

Statistic

Bands

How many

Cut points

Keep rows where

Value

and

Fill with

Value

Also mark which rows were filled

Rules

Base period = 100

Which part

Percent at each tail

Threshold

Survey weight optional

Direction

Names column

Values column

Largest first

Delete them, not just count them

Read this column as

Tidy the text

New name optional

EconStatsCraft rigorous, plain-spoken econometrics

Econometrics that explains itself.

Regression, panel data, time series and the whole battery of tests. Every result is followed by a verdict you can defend.

Load a dataset to see it here.

Survival analysis

What it needs

The intuition

When to use it

Where the estimator comes from

The model

What it assumes

Statistical properties

How to read the output

A worked reading

Symptom → diagnosis → what to do

Common mistakes

Where to read this properly

See also

Related Actuarial

All analyses

Charts

Maps

Explore

Regression

Time series - one series

Time series - several series

Panel data

Multivariate statistics

Marketing research

Causal

Finance & risk

Actuarial

Tests & breaks

Machine learning

Econometrics that explains itself.