Common mistake 1 with Unobserved components

The pile-up problem: variance estimates are bounded below by zero, and the maximum-likelihood estimate collapses to exactly zero far more often than it should. A zero variance is weak evidence that the component is truly fixed.

Common mistake 2 with Unobserved components

Reading the smoothed cycle as a real-time output gap. Revisions at the end of the sample are large - this is exactly the criticism levelled at HP-filtered output gaps.

Common mistake 3 with Unobserved components

Adding a cycle and a flexible trend at once. They compete to explain the same movement and the split between them is barely identified.

Common mistake 4 with Unobserved components

Forgetting that the end-of-sample estimates are the least reliable and are the ones the forecast is built on.

EconStatsCraft › Unobserved components

Unobserved components

Harvey’s structural approach: instead of differencing until the series behaves, model it as trend plus cycle plus season plus noise and get each piece back as something you can plot and interpret.

What it needs

Series

The intuition

A series is a level that wanders, plus a slope that may itself wander, plus a season, plus noise. An unobserved-components model says exactly that and lets the data decide how much each part moves. Unlike a moving average or an HP filter, it is a statistical model: the trend has a variance you can estimate and test, and if the data says the slope is constant the estimate comes back at zero rather than being imposed.

When to use it

When you want the components themselves, not just a forecast: the underlying trend in output, the cyclical position of the economy, a seasonally adjusted series. Also when you want a forecast whose assumptions are visible rather than buried in difference operators.

Where the estimator comes from

The local linear trend model is y_t=\mu_t+\gamma_t+\varepsilon_t,\quad \mu_t=\mu_{t-1}+\beta_{t-1}+\eta_t,\quad \beta_t=\beta_{t-1}+\zeta_t, with \gamma_t a seasonal component summing to approximately zero over a year. Each disturbance has its own variance, and the ratios among them - the SIGNAL-TO-NOISE ratios - determine how smooth each component is. Setting \mathrm{Var}(\eta)=0 gives a smooth trend with a wandering slope; setting \mathrm{Var}(\zeta)=0 gives a fixed slope. Estimation puts the whole thing in state-space form, runs the Kalman filter to build the likelihood, maximises it, and then runs the smoother to recover the components using the whole sample rather than just the past.

The model

y_t=\mu_t+\psi_t+\gamma_t+\varepsilon_t,\qquad \mu_{t+1}=\mu_t+\beta_t+\eta_t,\qquad \beta_{t+1}=\beta_t+\zeta_t

What it assumes

The disturbances driving each component are independent and Normal.
The components are additive - take logs first if they combine multiplicatively.
The chosen structure is right. A cycle you did not include shows up in the trend instead.
The sample is long enough to separate the components. Level, trend, seasonal and cycle all compete to explain the same variation, and in a short series the decomposition is driven by the priors and the parameterisation rather than by the data.

Statistical properties

The HP filter is a special case: a smooth trend model with the signal-to-noise ratio FIXED at 1/1600 rather than estimated. UCM estimates it, which is the substantive difference and usually the better idea.
The filtered estimate at time t uses data up to t; the smoothed estimate uses the whole sample. Smoothed components are much less jagged, and only the filtered ones are available in real time - a distinction that matters enormously for output-gap estimates.
A variance estimated at exactly zero is informative rather than a failure. It says that component does not move, and the model has simplified itself.
Components are identified by their dynamics, not by economics. A slowly moving trend and a very persistent cycle are close to indistinguishable in a short sample, and the split between them is fragile.
The Kalman filter handles missing observations natively - it simply skips the update step - which makes UCM a natural choice for ragged or irregular data.
Unlike the HP filter, the model produces standard errors for the components and genuine forecasts with intervals.

How to read the output

Each estimated VARIANCE says how much that component moves. A level variance near zero means the trend is effectively deterministic and a straight line would do.
A large level variance is the same message a unit root test gives, expressed as a component you can plot rather than a test you pass or fail.
The smoothed components use the whole sample. They are the best description of history, not what was knowable at the time - do not present them as real-time estimates.
A slope variance of zero with a non-zero level variance is a random walk with drift.

A worked reading

On a monthly series the model separates a smooth trend, a twelve-period seasonal and an irregular. The instructive comparison is filtered against smoothed: at the end of the sample they coincide, but in the middle the smoothed trend is visibly cleaner because it uses subsequent observations. An output gap computed from the smoothed trend therefore looks far more decisive than anything a policymaker could have known at the time - which is Orphanides' critique of real-time output gaps, visible directly in the two lines.

Symptom → diagnosis → what to do

If you see this	It means	Do this
A variance estimated at zero	That component is deterministic	Informative; simplify the specification
Trend absorbing the cycle	Signal-to-noise unidentified in this sample	Fix one variance or shorten the trend
Smoothed and filtered trends very different	Normal, and important	Only the filtered one existed in real time
Seasonal component drifting a lot	Seasonal variance too free	Constrain it, or use a fixed seasonal
Optimiser not converging	Too many free variances	Start from a simpler model and add components
Trend following every wiggle	Trend variance too high relative to noise	Check the estimated ratio against the HP convention

Common mistakes

The pile-up problem: variance estimates are bounded below by zero, and the maximum-likelihood estimate collapses to exactly zero far more often than it should. A zero variance is weak evidence that the component is truly fixed.
Reading the smoothed cycle as a real-time output gap. Revisions at the end of the sample are large - this is exactly the criticism levelled at HP-filtered output gaps.
Adding a cycle and a flexible trend at once. They compete to explain the same movement and the split between them is barely identified.
Forgetting that the end-of-sample estimates are the least reliable and are the ones the forecast is built on.

Where to read this properly

Harvey, Forecasting, Structural Time Series Models and the Kalman Filter (1989).
Durbin and Koopman, Time Series Analysis by State Space Methods (2012).
Orphanides and van Norden (2002), "The Unreliability of Output-Gap Estimates in Real Time", Review of Economics and Statistics 84(4).
Hamilton (2018), "Why You Should Never Use the Hodrick-Prescott Filter", REStat 100(5).

Related Time series - one series

Time-varying coefficients (Kalman) · Unit root tests · Correlogram (ACF / PACF) · ARIMA / SARIMA · Rolling-origin cross-validation · Theta method · STL forecasting · Forecast combination · Markov regime switching · Change frequency (resample) · ARCH / GARCH volatility · Decompose & filter · Spectral analysis · Exponential smoothing · Forecast accuracy

All analyses

Data & variables

Each example is simulated from a known model, so you can check whether the estimator recovers the truth - and nothing here is anyone's private data.

Filter variables

Drag a variable into a slot on the right - or just click it.

Choose an example dataset above to begin.

Choose a transformation - it opens in the data editor, with a live preview.

Apply to

Multiply by

Columns ctrl-click for several

Group by ctrl-click for several

Within groups

Statistic

Bands

How many

Cut points

Keep rows where

Value

and

Fill with

Value

Also mark which rows were filled

Rules

Base period = 100

Which part

Percent at each tail

Threshold

Survey weight optional

Direction

Names column

Values column

Largest first

Delete them, not just count them

Read this column as

Tidy the text

New name optional

EconStatsCraft rigorous, plain-spoken econometrics

Econometrics that explains itself.

Regression, panel data, time series and the whole battery of tests. Every result is followed by a verdict you can defend.

Load a dataset to see it here.

Unobserved components

What it needs

The intuition

When to use it

Where the estimator comes from

The model

What it assumes

Statistical properties

How to read the output

A worked reading

Symptom → diagnosis → what to do

Common mistakes

Where to read this properly

See also

Related Time series - one series

All analyses

Charts

Maps

Explore

Regression

Time series - one series

Time series - several series

Panel data

Multivariate statistics

Marketing research

Causal

Finance & risk

Actuarial

Tests & breaks

Machine learning

Econometrics that explains itself.