EconStatsCraft › Portfolio optimization

Portfolio optimization

Takes the sample means and covariance of several return series and finds the minimum-variance and maximum-Sharpe portfolios, traces the efficient frontier, and shows how far inside it every single asset sits.

What it needs

Asset returns - Two or more. Ctrl-click for several.

Frequently asked questions

What is portfolio optimization?

Portfolio optimization chooses the weights of assets to get the highest expected return for a given level of risk, or the lowest risk for a given return. In Markowitz mean-variance optimization, risk is the portfolio variance, which depends on every pair of assets' covariance, not just their individual volatilities.

What is the efficient frontier?

The efficient frontier is the curve of portfolios that offer the best possible expected return at each level of volatility. Any portfolio below it is inefficient: another mix of the same assets gives more return for the same risk.

What is the maximum Sharpe ratio portfolio?

It is the portfolio on the efficient frontier with the highest excess return per unit of volatility. With a risk-free asset, every investor should hold this one risky portfolio and adjust risk by mixing it with cash or borrowing, which traces out the capital market line.

What is the minimum variance portfolio?

The minimum variance portfolio is the leftmost point of the efficient frontier: the lowest-volatility mix of the assets, chosen without using expected returns at all. Because expected returns are the hardest input to estimate, it is often more stable out of sample than the maximum-Sharpe portfolio.

Why do optimized portfolio weights look extreme?

Mean-variance optimization treats estimated returns as exact, so it piles into assets whose returns happen to be overestimated. Constraining weights (no short selling, position limits), using longer data or shrinking the inputs gives more sensible weights.

The intuition

Given several assets, some combinations are simply better than others: same expected return for less risk. Plot every possible mix and the good ones trace out a curve, the efficient frontier. The mathematics is a constrained optimisation, but the economics is one idea - correlations below one mean a mix is less risky than its parts, and that free reduction in risk is the only free lunch in finance.

When to use it

Asset allocation, constructing a benchmark-agnostic portfolio, teaching diversification, or showing why holding many imperfectly-correlated assets beats holding the best one.

Where the estimator comes from

Minimise portfolio variance w'\Sigma w subject to w'\mu=\mu_p and w'\mathbf{1}=1. The Lagrangian gives a closed form: with A=\mathbf{1}'\Sigma^{-1}\mathbf{1}, B=\mathbf{1}'\Sigma^{-1}\mu and C=\mu'\Sigma^{-1}\mu, the minimum-variance frontier is the parabola \sigma_p^2=(A\mu_p^2-2B\mu_p+C)/(AC-B^2). The global minimum-variance portfolio is w=\Sigma^{-1}\mathbf{1}/A and needs no expected returns at all, which is why it is far more stable in practice. Adding a risk-free asset gives the tangency portfolio w\propto\Sigma^{-1}(\mu-R_f\mathbf{1}), the mix with the highest Sharpe ratio.

The model

\min_{w}\; w^{\top}\Sigma\,w \quad\text{s.t.}\quad w^{\top}\mathbf{1}=1,\; w^{\top}\mu = \bar{r};\qquad w_{\text{tan}} \propto \Sigma^{-1}(\mu - r_f\mathbf{1})

What it assumes

Returns are adequately described by their mean and covariance - no skew, no fat tails that matter, or at least that the investor only cares about mean and variance.
The sample mean and covariance are good estimates of the true ones. This is the weak point: sample means are extremely noisy.
No transaction costs or taxes; weights can be held exactly.
Short selling is permitted unless constrained, and the optimiser will use it. A weight of -45% is a genuine instruction to borrow and sell, not a rounding artefact.

Statistical properties

The optimiser is an error-maximiser. It puts the most weight on whichever asset's expected return was most over-estimated, because that is precisely what the objective rewards.
Expected returns are far harder to estimate than covariances. Merton (1980) showed that estimating a mean requires a long calendar span, not more frequent sampling; estimating a variance benefits from both.
The global minimum-variance portfolio uses no expected returns and consistently outperforms the full mean-variance solution out of sample for exactly that reason.
With N assets and T periods, the sample covariance matrix is singular whenever T is less than or equal to N, and badly conditioned well before that. Ledoit and Wolf (2004) shrinkage is the standard repair.
Unconstrained solutions routinely produce extreme long-short weights. A no-short constraint acts as an implicit shrinkage and usually improves out-of-sample results (Jagannathan and Ma, 2003).
DeMiguel, Garlappi and Uppal (2009) found that equal weighting beat mean-variance optimisation out of sample across many datasets. That is the benchmark any optimised portfolio should be made to beat.

How to read the output

Every individual asset plots inside the frontier - the gap between an asset and the curve directly above it is the diversification benefit.
The tangency portfolio is where a line from the risk-free rate just touches the frontier; it has the highest Sharpe ratio and, with a risk-free asset, everyone should hold some mix of it and cash (the capital market line).
The minimum-variance portfolio uses only the covariance, not the means, so it is far more stable out of sample - often it beats the tangency portfolio in practice.
Extreme or wildly shifting tangency weights are the signature of noisy expected returns, not of a good opportunity.

A worked reading

Five funds, and the global minimum-variance portfolio puts 100% of the money into one of them - the defensive fund, which happens to have the lowest volatility of the five (3.1691 against 5.17 to 6.47). That is the honest face of an unconstrained optimiser: nothing in the objective rewards diversification for its own sake, so if one asset dominates the covariance structure the solution is a corner. The resulting portfolio has an expected return of 0.5529 per period (6.635% annualised) at a volatility of 3.1691 (10.978% annualised). Notice what that means for the tangency portfolio further down, which adds expected RETURNS to the problem: minimum-variance uses only the covariance matrix, which is estimated reasonably well from 180 observations, while tangency also needs the means, which are not - the standard error on a mean return is roughly the volatility over the square root of the sample, and over any realistic sample that is large relative to the differences between assets. It is why minimum-variance weights are usually the ones worth taking to a client and tangency weights usually are not.

Symptom → diagnosis → what to do

If you see this	It means	Do this
Weights of +300% and -200%	Unconstrained optimiser on noisy means	Impose no-short, or use the minimum-variance portfolio
Weights change completely with one more month of data	Estimation error dominating	Shrink the covariance matrix; do not trust the point solution
One asset takes almost everything	Its sample mean is the highest, probably by luck	Check what happens with equal weights
Frontier is nearly a straight line	Assets are highly correlated	There is little diversification to be had here
Covariance matrix will not invert	More assets than periods	Shrinkage, or fewer assets
Tangency portfolio has a Sharpe ratio above 2	In-sample overfitting	This will not survive out of sample

Common mistakes

Trusting the tangency weights literally. Small changes in the estimated means move them a lot; many practitioners shrink the means, add constraints, or just use minimum-variance.
Optimising on a short sample where the covariance matrix is barely invertible.
Forgetting that the frontier is in-sample by construction - it will look worse on new data.
Reading a corner solution as a recommendation. An unconstrained minimum-variance optimiser will put everything in one asset whenever that asset dominates the covariance structure, and that is a statement about the estimates rather than about the market.

Where to read this properly

Markowitz (1952), "Portfolio Selection", Journal of Finance 7(1).
Merton (1980), "On Estimating the Expected Return on the Market", JFE 8(4) - why means are hard.
Ledoit and Wolf (2004), "Honey, I Shrunk the Sample Covariance Matrix", Journal of Portfolio Management 30(4).
DeMiguel, Garlappi and Uppal (2009), "Optimal Versus Naive Diversification", Review of Financial Studies 22(5).

Related Finance & risk

Black–Scholes option pricing · Bond price, duration & convexity · Merton distance to default · EMI / home loan planner · CAPM / beta estimation · Multi-factor model · Value at Risk & Expected Shortfall · Performance & drawdown · Time value of money & NPV/IRR · Equity valuation (DDM, FCF, residual income) · Cost of capital (WACC) · Binomial option pricing · Option strategies (payoff & profit) · Forwards, futures & swaps · Credit risk & Basel capital · Extreme value theory (tail risk)

All analyses

Data & variables

Each example is simulated from a known model, so you can check whether the estimator recovers the truth - and nothing here is anyone's private data.

Filter variables

Drag a variable into a slot on the right - or just click it.

Choose an example dataset above to begin.

Choose a transformation - it opens in the data editor, with a live preview.

Apply to

Multiply by

Columns ctrl-click for several

Group by ctrl-click for several

Within groups

Statistic

Bands

How many

Cut points

Keep rows where

Value

and

Fill with

Value

Also mark which rows were filled

Rules

Base period = 100

Which part

Percent at each tail

Threshold

Survey weight optional

Direction

Names column

Values column

Largest first

Delete them, not just count them

Read this column as

Tidy the text

New name optional

EconStatsCraft rigorous, plain-spoken econometrics

Econometrics that explains itself.

Regression, panel data, time series and the whole battery of tests. Every result is followed by a verdict you can defend.

Load a dataset to see it here.

Portfolio optimization

What it needs

Frequently asked questions

What is portfolio optimization?

What is the efficient frontier?

What is the maximum Sharpe ratio portfolio?

What is the minimum variance portfolio?

Why do optimized portfolio weights look extreme?

The intuition

When to use it

Where the estimator comes from

The model

What it assumes

Statistical properties

How to read the output

A worked reading

Symptom → diagnosis → what to do

Common mistakes

Where to read this properly

See also

Related Finance & risk

All analyses

Charts

Maps

Explore

Regression

Time series - one series

Time series - several series

Panel data

Multivariate statistics

Marketing research

Causal

Finance & risk

Actuarial

Tests & breaks

Machine learning

Econometrics that explains itself.