TL;DR: The probability of backtest overfitting (PBO), introduced by Bailey, Borwein, López de Prado and Zhu, estimates how often the configuration you would pick as best in-sample ends up below the median out of sample. A PBO near 0 means your selection process tends to find real winners; near 0.5 means it is no better than picking at random. It measures the selection process, not a single equity curve.
What problem does PBO solve?
Every strategy you see went through selection. You tried a few lookbacks, a couple of thresholds, maybe three models, and kept the one with the best backtest. The question that matters is not "is this backtest good?" but "does my way of choosing backtests tend to choose things that keep working?"
A single out-of-sample test cannot answer that. It is one draw. If you tried many configurations, the in-sample winner may simply be the one whose noise happened to line up with this particular history.
PBO turns that intuition into a number. David Bailey, Jonathan Borwein, Marcos López de Prado and Qiji Jim Zhu set out the method in The Probability of Backtest Overfitting, published in the Journal of Computational Finance. Their procedure is called combinatorially symmetric cross-validation (CSCV).
How is PBO calculated?
The recipe needs one input: a matrix of performance, with one column per configuration you tried and one row per time period.
- Build the matrix. For N configurations over T periods, record each configuration's returns.
- Split the rows. Partition the time periods into S equal, contiguous blocks, with S even.
- Form every half-and-half combination. Choose S/2 blocks as the in-sample set and the other S/2 as the out-of-sample set, for every possible combination. With S = 16 blocks there are 12,870 combinations.
- Select the in-sample winner. For each combination, compute every configuration's performance (for example Sharpe) on the in-sample blocks and pick the best.
- Rank it out of sample. Find where that winner ranks among all N configurations on the out-of-sample blocks, as a relative rank ω between 0 and 1.
- Take the logit. Compute λ = ln(ω / (1 − ω)). λ above zero means the winner beat the out-of-sample median; λ at or below zero means it did not.
- Count. PBO is the share of combinations where λ ≤ 0.
The method is "symmetric" because every block serves as in-sample and out-of-sample equally often, so no single arbitrary split drives the answer.
What else does the procedure show?
The same combinations give you more than one number. The paper also examines:
- Performance degradation: regress out-of-sample performance on in-sample performance across combinations. A negative slope means that stronger in-sample results predict weaker out-of-sample results, a hallmark of overfitting.
- Probability of loss: how often the selected configuration loses money out of sample.
- Stochastic dominance: whether picking the in-sample winner beats picking a configuration at random.
How should you read a PBO value?
| PBO | What it suggests |
|---|---|
| Near 0 | The in-sample winner usually stays in the top half out of sample. Selection is adding information. |
| Around 0.2 to 0.4 | Mixed. Some real ranking skill, but a meaningful share of splits where the pick disappoints. |
| Around 0.5 | Selection is roughly a coin flip. The in-sample ranking says little about the future. |
| Above 0.5 | Worse than random. The in-sample best tends to be among the worst out of sample, often a sign of heavy noise fitting. |
These bands are rules of thumb, not thresholds from the paper. Treat PBO as a probability estimate with its own sampling noise, especially with few configurations or short histories.
Why can't PBO be computed from one equity curve?
This is the most common misuse. PBO asks where the chosen configuration ranks among its competitors. If you only have one strategy's return series, there are no competitors to rank against. Splitting one curve into time blocks and treating the blocks as "configurations" produces a number, but it has no PBO interpretation.
To estimate PBO you need a family of alternatives evaluated on the same splits: parameter variants, model variants, or the full set of things you actually tried. The more honestly that set reflects your real search, the more useful the answer.
How is PBO different from the Deflated Sharpe Ratio?
Both come from the same research program on backtest overfitting, and they are complementary.
| Deflated Sharpe Ratio | Probability of Backtest Overfitting | |
|---|---|---|
| Question | Is this Sharpe still significant after my number of trials? | Does picking the in-sample best tend to pick a winner out of sample? |
| Input | One return series plus the trial count and trial dispersion | A matrix of performance across all configurations |
| Output | Probability the true Sharpe beats the best Sharpe expected from luck across the trials | Probability the in-sample winner underperforms the out-of-sample median |
| Main reference | Bailey and López de Prado (2014) | Bailey, Borwein, López de Prado and Zhu |
A strategy can pass one and fail the other. A high DSR with a high PBO suggests a strong result from a fragile process; a low DSR with a low PBO suggests a stable process that has not found much. Our Deflated Sharpe Ratio explainer covers the other half.
How does Trigr use PBO?
On Trigr, PBO is computed by Optimize with ML runs, whose results show it next to the Deflated Sharpe Ratio and out-of-sample Sharpe; the results treat a PBO of 50% or less as clearing the gate. A standard backtest does not compute PBO: its "Deep dive" section lists it as "Not yet computed" rather than inventing a value, consistent with the point above that one equity curve cannot produce a meaningful PBO. The backtesting docs cover how standard results are presented, and our guide on how to read a backtest walks through where PBO sits among the other numbers.
In ML optimization, described in the ML docs, PBO is estimated as follows:
- Combinatorial purged splits. Instead of plain CSCV blocks, the events are split with combinatorial purged cross-validation, so training samples whose labels overlap a test group are purged and embargoed. This matters because triple-barrier labels span several bars. The purged walk-forward guide explains purging.
- A small grid of competing configurations. For the model family that was selected, a handful of configurations are fit on each of up to ten splits. Each is scored by the Sharpe of the strategy returns it gates, in-sample and out-of-sample, and the logit-rank procedure above produces PBO.
- Diagnostic only. The PBO grid does not choose the model family, its parameters or the saved model. The family is chosen separately on anchored, purged walk-forward out-of-sample results, with fixed parameters and no hyperparameter tuning.
- Part of the leak verdict, as a soft flag. A PBO above 0.5, or a DSR below 0.60, is flagged in the verdict as a likely overfit. It stops the run from reading as a clean pass, but it is reported as weak evidence rather than as a leak.
It is worth being clear about the limits. With a small grid and a capped number of splits, the estimate is coarse. It measures overfitting risk within that grid, not across every idea you have tried outside the run, and separate optimizer reruns are not yet accumulated into it.
What should you do with a high PBO?
A high PBO is a reason to simplify, not to search harder.
- Reduce the search. Fewer parameters and fewer variants mean fewer ways to fit noise.
- Prefer robust regions. A configuration whose neighbors perform similarly is more trustworthy than a lonely peak.
- Get more data or longer horizons. More independent observations shrink the noise that selection exploits.
- Forward-test. A paper agent gives you an out-of-sample record that no amount of re-running can contaminate.
Backtests are not guarantees; perps are leveraged and can lose more than expected. PBO lowers the odds of fooling yourself, but it does not predict returns.
Next steps
See how PBO fits into the full research pipeline in machine learning for trading strategies without overfitting, or browse strategies with verified, versioned backtests on the Trigr marketplace.