Probabilistic Sharpe Ratio (PSR) and Backtest Overfitting
A statistically rigorous alternative to raw Sharpe ratio that adjusts for non-normality, sample length, skewness, and kurtosis to detect backtest overfitting.
This paper examines the Probabilistic Sharpe Ratio (PSR), a statistically rigorous extension of the classical Sharpe ratio that accounts for sample length, skewness, and kurtosis. PSR estimates the probability that an observed Sharpe ratio exceeds a benchmark, providing an inferential framework that penalizes short histories and non-normal return distributions. We present the mathematical formulation, a Python implementation, and discuss its application in detecting backtest overfitting.
Key Takeaways
- Raw Sharpe ratio says nothing about statistical confidence, non-normality, or multiple testing -- PSR addresses all three.
- PSR inflates uncertainty when returns are asymmetric or fat-tailed, penalizing strategies that benefit from favorable sampling noise.
- Ranking strategies by PSR instead of raw Sharpe forces a stronger evidentiary standard: short histories and ugly tail behavior are penalized.
- Standard Sharpe is descriptive; PSR is inferential -- it lets you test whether estimated skill exceeds a benchmark such as SR* = 1.
Introduction
The Sharpe ratio is one of the most abused statistics in quantitative finance. Two strategies can have the same Sharpe ratio even if one is estimated from a short, skewed, fat-tailed sample and the other from a long, well-behaved history. A raw Sharpe number says nothing about statistical confidence, non-normality, or multiple testing.
The Probabilistic Sharpe Ratio (PSR) estimates the probability that an observed Sharpe ratio exceeds a benchmark \(SR^*\), while adjusting for skewness and kurtosis:
Here, \(T\) is the sample length, \(\gamma_3\) is skewness, \(\gamma_4\) is kurtosis, and \(\Phi\) is the standard normal CDF. The denominator inflates uncertainty when returns are asymmetric or fat-tailed — exactly what classical Sharpe ignores.
Python Implementation
import numpy as np import pandas as pd from scipy.stats import skew, kurtosis, norm def probabilistic_sharpe_ratio(returns, sr_benchmark=0.0, periods_per_year=252): r = pd.Series(returns).dropna() sr_hat = np.sqrt(periods_per_year) * r.mean() / r.std(ddof=1) T = len(r) g3 = skew(r, bias=False) g4 = kurtosis(r, fisher=False, bias=False) # Pearson kurtosis numerator = (sr_hat - sr_benchmark) * np.sqrt(T - 1) denominator = np.sqrt(1 - g3 * sr_hat + ((g4 - 1) / 4.0) * sr_hat**2) z = numerator / denominator return { "sharpe": sr_hat, "psr": norm.cdf(z), "skew": g3, "kurtosis": g4, "z_score": z } # Example np.random.seed(42) rets = np.random.normal(0.0005, 0.01, 500) print(probabilistic_sharpe_ratio(rets, sr_benchmark=1.0))
Ranking strategies by PSR instead of raw Sharpe forces the strategy to "earn" its Sharpe under a stronger evidentiary standard. It penalizes short histories, punishes ugly tail behavior, and gives you a way to compare estimated skill against a benchmark such as \(SR^* = 1\). Standard Sharpe is descriptive. PSR is inferential.
Assumptions
- Returns are independent. PSR corrects for skewness and kurtosis but assumes serial independence. Autocorrelated returns — which is what smoothed or illiquid marks produce — understate the true standard error and inflate PSR.
- The benchmark is chosen before looking. PSR asks whether the true Sharpe exceeds SR*. Setting SR* to zero after seeing the result is the most common way the statistic is abused.
- The track record is the whole track record. The formula takes n at face value. If the sample begins after a bad year was dropped, no correction can recover it.
- One strategy, one test. PSR says nothing about how many variants were tried. That is what the Deflated Sharpe Ratio addresses.
Robustness: what would change the conclusion
Trial count dominates everything else. The same Sharpe of 1.50 gives PSR 1.0000 against a zero benchmark, DSR 0.8736 after 100 trials, and DSR 0.2671 after 1,000. Nothing about the strategy changes between those numbers — only the honesty of the accounting. A PSR quoted without a trial count is close to meaningless.
Kurtosis matters more than intuition suggests. The kurtosis
term does not vanish at the normal value of 3. A published worked example on
this page previously got that wrong, giving denominators of 1.000 and 2.318
against the correct 1.4577 and 2.4850; the correction is disclosed rather than
silently patched. Every intermediate value is now shown so the arithmetic can
be checked by hand or against scipy.
Short records cannot be rescued. The √(n−1) factor means a high Sharpe over a few dozen observations stays statistically weak no matter how attractive the point estimate is.
Gross versus net Sharpe
PSR is a statement about the return series you feed it, and nothing more. A gross-of-cost series produces a gross-of-cost PSR, and the gap between the two widens with turnover: a high-frequency strategy can carry a comfortable gross PSR and a negative net Sharpe.
Before computing PSR, subtract realistic costs — spread, market impact and delay — using something like the square-root impact model. Applying a statistical significance test to a return stream that could not have been captured is precision applied to the wrong quantity.
Limitations
- Does not correct for selection bias. PSR evaluates one track record in isolation. If the strategy is the best of many tested, PSR overstates the evidence and the Deflated Sharpe Ratio should be used instead.
- Assumes IID returns. Serial correlation, common in illiquid or smoothed portfolios, inflates the observed Sharpe and is not handled by the standard formulation.
- Higher moments are themselves estimates. Skewness and kurtosis from short samples are noisy, and the correction is only as reliable as those estimates.
- Says nothing about economics. PSR is a statistical statement about a return series. It does not address costs, capacity, regime change, or whether the strategy was fitted to its sample.
References
- Bailey, D. & López de Prado, M. (2012). “The Sharpe Ratio Efficient Frontier.” Journal of Risk 15(2), 3–44.
- Bailey, D. & López de Prado, M. (2014). “The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting and Non-Normality.” Journal of Portfolio Management 40(5), 94–107.
- Lo, A. (2002). “The Statistics of Sharpe Ratios.” Financial Analysts Journal 58(4), 36–52.
- Harvey, C. & Liu, Y. (2015). “Backtesting.” Journal of Portfolio Management 42(1), 13–28.
QuantMedia research is independent and not peer reviewed. These references are the primary sources the analysis draws on; readers are encouraged to consult them directly rather than relying on this summary.
Research record
- Author
- Cemil Ertürk · QuantMedia Research
- Published
- January 25, 2026
- Last material revision
- August 16, 2026
- Research version
- 1.1
- Topic
- Statistics
- Code
- Runnable implementation with tests
- Review status
- Independent research. Not peer reviewed.
How to cite this research
Cemil Ertürk. "Probabilistic Sharpe Ratio (PSR) and Backtest Overfitting." QuantMedia, 2026. https://quantmedia.io/paper-probabilistic-sharpe-ratio.html
BibTeX
@misc{erturk2026probabilistic,
author = {Ert{\"u}rk, Cemil},
title = {Probabilistic Sharpe Ratio (PSR) and Backtest Overfitting},
year = {2026},
howpublished = {QuantMedia},
url = {https://quantmedia.io/paper-probabilistic-sharpe-ratio.html},
note = {Accessed: <date>}
}
Code & Reproducibility
The measure described above is implemented as an interactive calculator that runs entirely in your browser — nothing is uploaded.
| Tool | Probabilistic Sharpe Ratio calculator |
|---|---|
| Inputs | Observed Sharpe, benchmark Sharpe, observations, skewness, kurtosis |
| Verification | A worked example is published with every intermediate value (denominator 2.4850, z = 2.8955, PSR = 0.9981) so the tool can be checked against an independent implementation |
| Explainer | What is the Probabilistic Sharpe Ratio? |
No standalone Python package is published for PSR: the formula is a single expression and the calculator already exposes every intermediate term. If you are selecting the best of many backtests, PSR is the wrong tool — the Deflated Sharpe Ratio adjusts for the number of trials.