Common mistake 1 with Buhlmann credibility

Applying credibility to risks that are not exchangeable - mixing a hazardous class with a benign one and blending both toward a meaningless average.

Common mistake 2 with Buhlmann credibility

Using the sample mean of a tiny cell as if k were zero; that is exactly the over-reaction credibility exists to prevent.

Common mistake 3 with Buhlmann credibility

Estimating VHM from a handful of risks and treating the resulting Z as precise.

Common mistake 4 with Buhlmann credibility

Applying the formula to claim counts and claim amounts interchangeably. The variance structure differs, so k differs, and a credibility factor computed on frequency does not transfer to severity.

EconStatsCraft › Buhlmann credibility

Buhlmann credibility

Blends each risk’s own claims experience with the average across all risks. Estimates the expected process variance and the variance of hypothetical means, forms the credibility factor Z for each risk, and reports the credibility premium against the raw mean and the grand mean.

What it needs

Risk identifier (policyholder, class, region)
Claim / loss amount

The intuition

A single class of business has its own claims history, but if that history is short it is mostly noise. The book as a whole has far more data but is not specific to this class. Credibility theory answers the obvious question - how much weight should the class's own experience get - and the answer is a formula rather than a judgement: more weight when you have more years, and more weight when classes genuinely differ from each other.

When to use it

Experience rating in general insurance and reinsurance, setting class or territory relativities, workers' compensation and group health pricing, or any time you must price a small cell without over-reacting to its short record.

Where the estimator comes from

Buhlmann\'s model treats each risk as drawing a latent parameter \Theta from a population. Define the expected process variance s^2=E[\mathrm{Var}(X\mid\Theta)] - noise WITHIN a risk - and the variance of the hypothetical means \tau^2=\mathrm{Var}(E[X\mid\Theta]) - genuine spread BETWEEN risks. The linear estimator minimising mean squared error is \hat{\mu}=Z\bar{X}+(1-Z)\mu with Z=\frac{n}{n+k},\qquad k=\frac{s^2}{\tau^2}. Read k as the number of years of own experience needed before the class\'s own mean deserves half the weight. When risks are nearly identical \tau^2\to0, k\to\infty and Z\to0: the collective mean is used. When they differ wildly, Z\to1.

The model

Z = \frac{n}{n+k},\qquad k = \frac{\text{EPV}}{\text{VHM}},\qquad P_{\text{cred}} = Z\,\bar{X}_i + (1-Z)\,\mu

What it assumes

Risks are exchangeable within the portfolio - drawn from a common distribution of hypothetical means - and their experience is independent across risks and over time.
The process variance is the same for every risk (Buhlmann); Buhlmann-Straub relaxes this to a common variance per unit of exposure.
The variance components EPV and VHM are estimated, not known, so k and Z carry their own sampling error - especially with few risks.
The collective mean is estimated from a book that is relevant to the risk being priced. Credibility shrinks towards the book average, so a book containing very different business shrinks a class towards a number that was never its target.

Statistical properties

Z rises with n but never reaches 1 in finite samples, so the collective always retains some weight. That is the shrinkage, and it is what makes the estimator beat the raw class mean in mean squared error.
The credibility formula is exactly a Bayesian posterior mean under a normal-normal model, and is the best LINEAR approximation to it in general. Buhlmann derived it without needing the distribution.
k has a clean reading: the number of exposure units at which Z = 0.5. Quoting k is more informative than quoting Z at one sample size.
Buhlmann-Straub extends this to unequal exposures, replacing n with the exposure measure. Almost all practical work uses that version.
Estimating s-squared and tau-squared from the data makes this empirical Bayes, and the resulting tau-squared estimate can come out negative in small samples. It is truncated at zero, which biases Z downward.
The same shrinkage logic appears as the James-Stein estimator and as a random-effects model. They are three names for one idea.

How to read the output

k is the number of observations at which a risk earns 50% credibility. A small k means the risks are genuinely different and their own data matters quickly.
The shrinkage plot shows every risk's raw mean pulled toward the grand mean. Thin risks are pulled almost all the way; data-rich risks barely move.
A negative raw estimate of VHM (shown as a small floor) means the between-risk differences are swamped by noise - the data cannot tell the risks apart, so Z should be near zero.
The classical full-credibility standard is the alternative framing: how many expected claims are needed before a risk's own mean is used with no blending.

A worked reading

Eight classes with a grand mean claim of 512.65. The variance decomposition is what drives everything: expected process variance (the noise within a class) is 299,790, while the variance of hypothetical means (the real differences between classes) is only 9,077. So k = EPV/VHM = 33.03, and with exposures between 32 and 66 the credibility factors land between 0.496 and 0.667 - roughly half to two-thirds weight on a class's own experience. Class A illustrates the consequence: its own mean is 337.74, far below the book, but with Z = 0.553 it is charged 415.96 rather than 337.74, because most of the gap is judged to be noise rather than a genuinely better risk. The classical alternative asks for about 2,317 expected claims before granting full credibility, which no class here comes close to - and that is the honest position, since a k of 33 says these classes are far more alike than the year-to-year noise within them.

Symptom → diagnosis → what to do

If you see this	It means	Do this
Z near zero with plenty of data	tau-squared small: classes are alike	Correct behaviour. The classification may not be earning its keep
Z near one immediately	Classes differ far more than they vary internally	Check the classes are not simply mislabelled
Negative tau-squared estimate	Sampling noise exceeds between-class variance	Truncate at zero and report that it happened
Credibility premium outside the range of the two inputs	An arithmetic error: it is a weighted average	Z must lie in [0,1]
Very unequal exposures across classes	Plain Buhlmann is the wrong model	Use Buhlmann-Straub
One class dominating the collective mean	The collective is not a fair prior for the others	Weight the collective by exposure

Common mistakes

Applying credibility to risks that are not exchangeable - mixing a hazardous class with a benign one and blending both toward a meaningless average.
Using the sample mean of a tiny cell as if k were zero; that is exactly the over-reaction credibility exists to prevent.
Estimating VHM from a handful of risks and treating the resulting Z as precise.
Applying the formula to claim counts and claim amounts interchangeably. The variance structure differs, so k differs, and a credibility factor computed on frequency does not transfer to severity.

Where to read this properly

Buhlmann (1967), "Experience Rating and Credibility", ASTIN Bulletin 4(3).
Buhlmann and Straub (1970), on unequal exposures.
Klugman, Panjer and Willmot, Loss Models (2019), ch. 16 and 17.
Efron and Morris (1977), "Stein's Paradox in Statistics", Scientific American - the same shrinkage, explained without insurance.

Related Actuarial

Life table · Life annuity · Life insurance pricing · Chain ladder reserving · Survival analysis · Mortality graduation · Aggregate loss model · Ruin probability · Multi-state Markov model · Loss severity fitting

All analyses

Data & variables

Each example is simulated from a known model, so you can check whether the estimator recovers the truth - and nothing here is anyone's private data.

Filter variables

Drag a variable into a slot on the right - or just click it.

Choose an example dataset above to begin.

Choose a transformation - it opens in the data editor, with a live preview.

Apply to

Multiply by

Columns ctrl-click for several

Group by ctrl-click for several

Within groups

Statistic

Bands

How many

Cut points

Keep rows where

Value

and

Fill with

Value

Also mark which rows were filled

Rules

Base period = 100

Which part

Percent at each tail

Threshold

Survey weight optional

Direction

Names column

Values column

Largest first

Delete them, not just count them

Read this column as

Tidy the text

New name optional

EconStatsCraft rigorous, plain-spoken econometrics

Econometrics that explains itself.

Regression, panel data, time series and the whole battery of tests. Every result is followed by a verdict you can defend.

Load a dataset to see it here.

Buhlmann credibility

What it needs

The intuition

When to use it

Where the estimator comes from

The model

What it assumes

Statistical properties

How to read the output

A worked reading

Symptom → diagnosis → what to do

Common mistakes

Where to read this properly

See also

Related Actuarial

All analyses

Charts

Maps

Explore

Regression

Time series - one series

Time series - several series

Panel data

Multivariate statistics

Marketing research

Causal

Finance & risk

Actuarial

Tests & breaks

Machine learning

Econometrics that explains itself.