Common mistake 1 with Key driver analysis

Asking more questions about one construct than another. The construct with more indicators looks more important, and that is a property of the questionnaire - the survey sample in this app has it built in so it can be seen.

Common mistake 2 with Key driver analysis

Reading a small weight as "do not invest". A driver everyone already rates highly has little variance left to explain the outcome, so it can matter enormously and score low. Cross the importance with the performance level before deciding.

Common mistake 3 with Key driver analysis

Treating the ranking as stable. Bootstrap it - with correlated drivers the order of the middle ranks is often within noise.

Common mistake 4 with Key driver analysis

Presenting the ranking without the confidence intervals. Adjacent drivers are routinely within sampling error of each other, and a chart that orders them implies a precision the data does not have.

EconStatsCraft › Key driver analysis

Key driver analysis

Which drivers really move satisfaction, NPS or intent. Regression coefficients are the wrong answer here because survey drivers are always correlated, so they split a shared contribution arbitrarily; relative weights apportion it properly and sum to R².

What it needs

Outcome - Overall satisfaction, NPS, repurchase intent.
Drivers - The attribute ratings you think move it.

The intuition

Which parts of the experience actually drive overall satisfaction? Regressing satisfaction on its components sounds simple and is treacherous: the components are all correlated with one another, so ordinary coefficients bounce around, sometimes turn negative, and cannot be read as importance. Relative importance methods split the explained variance among correlated predictors in a way that is stable and sums to the total.

When to use it

Whenever you have a battery of attribute ratings and one overall measure, and the question is where to invest. It is the standard answer in customer experience and brand tracking work.

Where the estimator comes from

With correlated predictors, the R^2 of a regression cannot be attributed to individual variables by their coefficients. SHAPLEY VALUE decomposition takes the average marginal contribution of each predictor over all 2^k orderings: \phi_j=\sum_{S\subseteq N\setminus\{j\}}\frac{|S|!\,(k-|S|-1)!}{k!}\left[R^2(S\cup\{j\})-R^2(S)\right], which is the unique attribution satisfying efficiency (the shares sum to R^2), symmetry and the null-player property. Johnson\'s relative weights approximate the same thing far more cheaply by orthogonalising the predictors, regressing on the orthogonal set, and mapping the weights back - which is why it is what most software actually runs.

The model

\varepsilon_j=\sum_{k}\Lambda_{jk}^{2}\,\beta_k^{*2},\qquad \sum_j \varepsilon_j=R^2

What it assumes

That the drivers are the right ones. Relative weights apportion the variance among the drivers you supplied - an omitted driver’s importance is silently redistributed among its correlates.
A linear, additive relationship, as in the regression underneath it.
That association is what you want. This is cross-sectional survey data: acting on it assumes the association would survive the intervention, which is a causal claim the data cannot support.
That the outcome and the drivers were measured in the same survey, on the same respondents, at the same time - and therefore share whatever mood, order and acquiescence effects that survey induced. Common-method variance inflates every weight together.

Statistical properties

Relative weights sum to the model R-squared, so they can be reported as percentages of explained variance. A raw regression coefficient cannot be.
They are always non-negative, which is a feature for communication and a limitation for inference - a genuinely harmful driver is not representable.
Correlated predictors share credit rather than one winning. Two nearly identical service measures each get about half the importance instead of one taking it all and the other appearing irrelevant.
Importance is not causality. A driver can rank high because it is a consequence of overall satisfaction rather than a cause - "would recommend" is the usual offender and should be excluded.
Shapley is exact but costs 2 to the k regressions, which is prohibitive beyond about twenty predictors. Johnson's relative weights approximate it closely at negligible cost.
A quadrant chart plotting importance against current performance is the standard output, and its usefulness depends entirely on the performance axis being measured on a comparable scale across drivers.

How to read the output

Read the shares of R² - they add to 100% of what the model explains, so they are directly comparable across drivers.
Compare the weights with the betas. Where they disagree, the drivers are correlated and the betas have split a shared contribution arbitrarily; the weights are the ones to act on.
Check R² before the ranking. Apportioning 15% of the variance precisely is still apportioning 15%.
Relative weights are shares of EXPLAINED variance. Two studies with different R squared cannot have their weights compared, because they are shares of different quantities.

A worked reading

Six service attributes, all correlated at 0.4 to 0.7, regressed on overall satisfaction. OLS gives one negative coefficient on an attribute nobody believes is harmful - the classic multicollinearity symptom. Relative weights give six non-negative shares summing to the model R-squared of 0.58, with the top driver at 0.17 and the bottom at 0.04. The ranking is stable across bootstrap resamples where the OLS coefficient signs are not, which is the practical case for the method.

Symptom → diagnosis → what to do

If you see this	It means	Do this
A negative OLS coefficient on an obviously good attribute	Multicollinearity	The reason to use relative importance at all
One driver with almost all the importance	It may be a restatement of the outcome	Check for near-duplicates of the dependent variable
Importances summing to well under 1	They sum to R-squared, not to 1	Normalise if reporting shares, and say which you did
Rankings unstable across samples	Highly correlated drivers	Report bootstrap intervals on the importances
Low overall R-squared	The drivers do not explain satisfaction	Importance shares of a small R-squared mean little
Acting on importance alone	Ignores current performance	Use the importance-performance quadrant

Common mistakes

Asking more questions about one construct than another. The construct with more indicators looks more important, and that is a property of the questionnaire - the survey sample in this app has it built in so it can be seen.
Reading a small weight as "do not invest". A driver everyone already rates highly has little variance left to explain the outcome, so it can matter enormously and score low. Cross the importance with the performance level before deciding.
Treating the ranking as stable. Bootstrap it - with correlated drivers the order of the middle ranks is often within noise.
Presenting the ranking without the confidence intervals. Adjacent drivers are routinely within sampling error of each other, and a chart that orders them implies a precision the data does not have.

Where to read this properly

Johnson (2000), "A Heuristic Method for Estimating the Relative Weight of Predictor Variables", Multivariate Behavioral Research 35(1).
Gromping (2007), "Estimators of Relative Importance in Linear Regression", The American Statistician 61(2).
Shapley (1953), "A Value for n-Person Games", Contributions to the Theory of Games II.
Lipovetsky and Conklin (2001), "Analysis of Regression in Game Theory Approach", Applied Stochastic Models 17(4).

Related Marketing research

Conjoint analysis · MaxDiff (best–worst scaling) · TURF - reach of a product line · Price sensitivity (Van Westendorp) · Market basket analysis · Perceptual map · Scale reliability (Cronbach’s alpha)

All analyses

Marketing research

Causal

Spatial autocorrelation · Spatial regression · Heckman selection · Instrumental variables (2SLS) · Difference-in-differences · Staggered difference-in-differences · Regression discontinuity · Matching & weighting · Oaxaca–Blinder decomposition

Finance & risk

Black–Scholes option pricing · Bond price, duration & convexity · Merton distance to default · EMI / home loan planner · CAPM / beta estimation · Multi-factor model · Portfolio optimization · Value at Risk & Expected Shortfall · Performance & drawdown · Time value of money & NPV/IRR · Equity valuation (DDM, FCF, residual income) · Cost of capital (WACC) · Binomial option pricing · Option strategies (payoff & profit) · Forwards, futures & swaps · Credit risk & Basel capital · Extreme value theory (tail risk)

Actuarial

Life table · Life annuity · Life insurance pricing · Chain ladder reserving · Buhlmann credibility · Survival analysis · Mortality graduation · Aggregate loss model · Ruin probability · Multi-state Markov model · Loss severity fitting

Tests & breaks

Monte Carlo experiment · F-test, Wald & Chow · Structural breaks & stability

Machine learning

Regression vs machine learning · Ridge / Lasso / Elastic Net · Random forest · Gradient boosting

Open the workspace →

Skip to results

Data & variables

Each example is simulated from a known model, so you can check whether the estimator recovers the truth - and nothing here is anyone's private data.

Filter variables

Drag a variable into a slot on the right - or just click it.

Choose an example dataset above to begin.

Choose a transformation - it opens in the data editor, with a live preview.

Apply to

Multiply by

Columns ctrl-click for several

Group by ctrl-click for several

Within groups

Statistic

Bands

How many

Cut points

Keep rows where

Value

and

Fill with

Value

Also mark which rows were filled

Rules

Base period = 100

Which part

Percent at each tail

Threshold

Survey weight optional

Direction

Names column

Values column

Largest first

Delete them, not just count them

Read this column as

Tidy the text

New name optional

EconStatsCraft rigorous, plain-spoken econometrics

Econometrics that explains itself.

Regression, panel data, time series and the whole battery of tests. Every result is followed by a verdict you can defend.

Load a dataset to see it here.

Key driver analysis

What it needs

The intuition

When to use it

Where the estimator comes from

The model

What it assumes

Statistical properties

How to read the output

A worked reading

Symptom → diagnosis → what to do

Common mistakes

Where to read this properly

See also

Related Marketing research

All analyses

Charts

Maps

Explore

Regression

Time series - one series

Time series - several series

Panel data

Multivariate statistics

Marketing research

Causal

Finance & risk

Actuarial

Tests & breaks

Machine learning

Econometrics that explains itself.