Machine Learning Trading Strategies Without Overfitting

How to build a machine learning trading strategy that survives out of sample: point-in-time features, triple-barrier labels, purged validation, DSR and PBO.

Trigr ResearchUpdated 9 min read
On this page
  1. Why do most machine learning trading strategies fail out of sample?
  2. What does a leak-resistant ML pipeline look like?
  3. How does Trigr's ML optimization work end to end?
  4. What does the leak verdict check?
  5. What are the honest limits?
  6. How do I get started?
  7. Next steps

TL;DR: A machine learning trading strategy avoids overfitting through process, not model choice: features that only use information available at decision time, labels that reflect how a trade actually resolves, validation that purges overlapping samples, and metrics that deflate for the number of things you tried. The methods most practitioners use come from Marcos López de Prado's Advances in Financial Machine Learning; this guide explains them and how Trigr applies them, including what it deliberately does not do.

Why do most machine learning trading strategies fail out of sample?

Financial data is short, noisy and non-stationary. A few years of hourly bars sounds like a lot of rows, but the number of independent outcomes is much smaller, and the signal-to-noise ratio is tiny compared with image or text problems. A flexible model will happily memorize that noise.

Three failure modes account for most broken results:

  • Leakage. Some information from the future reaches the training data or the features. Standard k-fold cross-validation shuffles time, and labels that span several bars overlap the test window. The model looks brilliant and trades badly.
  • Selection bias. You try fifty feature sets, ten models and several label horizons, then report the best. Even if every variant is noise, the best of many noisy results looks like skill.
  • Mismatch between label and trade. The model predicts "next bar up or down" while the strategy holds for a day with a stop-loss. The metric the model optimizes is not the one your account experiences.

None of these is fixed by a better algorithm. They are fixed by the research pipeline around the algorithm, which is the core argument of López de Prado's Advances in Financial Machine Learning (Wiley, 2018).

What does a leak-resistant ML pipeline look like?

A sound pipeline has five stages. Each one closes a specific hole.

Stage What it prevents Typical method
Point-in-time features Look-ahead bias in inputs Every feature uses only data available at the bar's close
Labels Label/trade mismatch Triple-barrier labels; optionally meta-labels
Sample weights Overcounting overlapping outcomes Average uniqueness of each label
Validation Leakage across the train/test boundary Purged, embargoed walk-forward or combinatorial CV
Evaluation Selection bias and luck Deflated Sharpe Ratio, PBO, leak controls

The steps below walk through each stage; sibling articles cover each in depth.

Step 1: Point-in-time features

Every feature must be computable at the moment the decision is made. That sounds obvious, yet it is broken constantly: a daily indicator joined onto hourly bars using the day's final value, a funding rate stamped at settlement but used before it, or a macro series aligned to its reference date instead of its release date.

The rule is simple to state: at bar t, a feature may use information from bar t's close and earlier, and slower inputs are carried forward from their last known value, never read ahead. Our point-in-time backtesting guide covers the data side in detail.

In Trigr, the ML feature loader enforces this rather than trusting it. It raises an error on any future leakage it detects and drops the traded bar's close from the inputs, because a strategy that fills at the next bar's open should not be learning from a price it could not have acted on.

Step 2: Labels that match how trades resolve

The fixed-horizon label ("return over the next n bars is positive") ignores the path. A trade that hits its stop on bar two and then recovers is labeled a winner. López de Prado's triple-barrier method fixes this: for each event, place a profit-taking barrier above, a stop-loss barrier below and a time barrier ahead, and label the event by whichever is touched first. Barrier widths scale with recent volatility, so a quiet market and a volatile one produce comparable labels.

A second idea, meta-labeling, changes the question entirely. Instead of asking the model to predict direction, you let a primary strategy decide the side, and train a secondary model to predict whether each of those trades will work. That turns a hard problem (forecast the market) into an easier one (filter a strategy's own signals).

Trigr's Studio optimizer uses exactly this setup: triple-barrier labels scaled by volatility, applied to your strategy's own entries, so the model learns a take-or-skip gate. The triple-barrier and meta-labeling article explains both in depth.

Step 3: Weight samples by uniqueness

Labels that span many bars overlap. If ten consecutive events all resolve on the same rally, a model trained on them sees one market move counted ten times. López de Prado's answer is average uniqueness: for each label, average the inverse of how many labels are active on each bar of its life. Heavily overlapping labels get small weights; isolated ones get full weight.

Trigr computes sample weights as average uniqueness multiplied by the absolute return of each label, so redundant events and trivially small moves both count for less. The same uniqueness adjustment is used when estimating the effective sample size for the Sharpe statistics, which keeps overlapping events from inflating confidence.

Step 4: Validation that respects time

This is where most retail ML goes wrong. Shuffled k-fold cross-validation trains on the future to predict the past. Even an unshuffled split leaks when a training label's horizon extends into the test window.

The fix has two parts:

  • Purging removes any training event whose label interval overlaps the test period.
  • Embargo adds a gap next to the test period, so serial correlation just outside the boundary cannot carry information across.

In a walk-forward setup you train only on the past, test on the next block, then roll forward. Trigr uses anchored (expanding), purged walk-forward splits to produce out-of-sample predictions, with an embargo sized to be at least the label horizon. The out-of-sample predictions are what decide which model family wins. Read the purged walk-forward validation guide for a worked example and the difference from scikit-learn's TimeSeriesSplit.

Step 5: Deflate the result for everything you tried

A single out-of-sample Sharpe ratio is still a sample statistic. If you ran many trials, the expected maximum of those trials is well above zero even with no edge. Two tools address this:

  • The Deflated Sharpe Ratio (Bailey and López de Prado, 2014) is the probability that the true Sharpe exceeds the best Sharpe you would expect from luck across your trials, accounting for the sample length, skewness and fat tails. See the Deflated Sharpe Ratio explainer.
  • The Probability of Backtest Overfitting (Bailey, Borwein, López de Prado and Zhu) estimates how often the configuration that looked best in-sample lands in the bottom half out of sample. The original PBO paper describes the combinatorial procedure, and our PBO article explains how to read it.

Trigr reports the DSR using the run's own classifier trial count, and a PBO estimated from a separate combinatorial purged cross-validation grid.

How does Trigr's ML optimization work end to end?

"Optimize with ML" in Studio takes a strategy you already built and learns an entry gate for it. The full pipeline, as documented on the AI and ML docs page:

  1. Features are built point-in-time from the strategy's own graph inputs (indicators, funding, open interest and other data sources it uses). The loader raises on future leakage.
  2. Labels are volatility-scaled triple-barrier labels on the strategy's entries, measured from the next bar's open, matching how the backtester fills.
  3. Sample weights come from average uniqueness and return attribution.
  4. Model family selection: LightGBM, random forest and XGBoost each run with fixed, disclosed parameters. The family with the best anchored, purged walk-forward out-of-sample result is selected. Hyperparameters are not tuned.
  5. Diagnostics: out-of-sample Sharpe, DSR, PBO from a separate CPCV grid (diagnostic only; it does not choose the model or its parameters), and feature importance by MDA, SFI and SHAP.
  6. Leak verdict: a realistic or not-realistic call with specific flags.
  7. Comparison: the strategy's entries gated by the out-of-sample predictions, shown against the baseline with the difference.

The selected family is then refit on the full training window and saved. Live inference loads that saved model each bar with the same point-in-time discipline and does not retrain; a later refit is a new, metered training run. Each run carries a surcharge of about 200 credits plus model cost.

What does a run give back?

Read the outputs in this order, starting with the verdict, not the equity curve.

Output What it tells you Read it as
Leak verdict and flags Whether the pipeline found signs of contamination A gate: if it says not realistic, stop there
Out-of-sample Sharpe Performance of the gated strategy on walk-forward test folds A sample estimate, not a forecast
Deflated Sharpe Ratio Probability the true Sharpe beats the luck benchmark for this run's trials The headline significance number; the results treat 0.60 as the research floor
PBO How often the in-sample favorite underperforms out of sample A diagnostic on selection risk
Feature importance (MDA, SFI, SHAP) Which inputs the model relies on, in and out of sample A research map and a leak check
Baseline vs gated backtest What the gate changed relative to your original entries Context, with trade counts in view

The DSR follows the method in Bailey and López de Prado's paper, with the sample size adjusted for overlapping labels. A gated backtest that trades far less than the baseline can show a better Sharpe simply by sitting out; check how many trades survived the gate before drawing conclusions.

Why no hyperparameter tuning?

Tuning is a search, and every search adds trials that the statistics must pay for. On short financial samples, a grid over depth, learning rate and leaf counts mostly finds the settings that fit this particular history's noise. Fixing the parameters keeps the number of trials small and honest, and it keeps the DSR's trial count meaningful. A tuned model might sometimes do better; we chose a smaller claim we can defend.

What does the leak verdict check?

Prevention is only the first line. Trigr's leak battery actively tries to prove a result is contaminated, and it will not call a run realistic if any hard check fails:

  • Shuffled-label control. Retrain on permuted labels. Accuracy must collapse to within 0.10 of chance (for an imbalanced meta-label, chance is the majority-class rate, not 50%). If a model trained on nonsense still beats chance, information is leaking through the pipeline.
  • Embargo sensitivity. Widen the embargo. If out-of-sample performance falls below half of its original level, the result is probably exploiting label overlap.
  • Single-feature dominance. If one feature carries more than 60% of the total permutation importance (MDA), it is often a future-derived or label-correlated column.
  • Implausibility ceiling. Directional accuracy above 0.65 (or, for a meta-label gate, more than 0.15 above the majority-class rate), or an out-of-sample Sharpe far above the plain strategy's own backtest Sharpe, is treated as too good to be real.

Two further checks are soft warnings, marking weak or thin evidence rather than a leak:

  • DSR and PBO. A Deflated Sharpe Ratio below 0.60 or a PBO above 0.5 is flagged as a likely overfit rather than a leak.
  • Low power. Fewer than 30 pooled out-of-sample events, or fewer than three purged folds, is flagged as indicative rather than conclusive.

What are the honest limits?

Even a clean pipeline leaves open questions, and it is better to know them up front:

  • Repeated runs are not yet accumulated. Each optimization's DSR counts that run's trials. If you run the optimizer twenty times on the same strategy and data, the twentieth result has been selected from twenty, and the statistics do not know it. Repeated tuning needs a fresh holdout or live validation.
  • The backtest's out-of-sample split is not the ML validation. Studio holds out the last 30% of a standard backtest by default, and it is unseen only until you act on it. The ML out-of-sample figures come from the walk-forward folds, not that split.
  • PBO is a diagnostic over a small grid. It estimates overfitting risk for the chosen model family; it is not a proof of robustness.
  • Regimes change. A model validated across 2021 to 2025 can still meet a market it has never seen.

Backtests are not guarantees; perps are leveraged and can lose more than expected. Forward-test on a paper agent before committing capital.

How do I get started?

Build or pick a strategy with a clear entry rule, run a standard backtest to see the net result, then run ML optimization and read the leak verdict before the headline number. You can drive the same workflow from ChatGPT, Claude or Codex by connecting your AI assistant over MCP, where ML optimization runs asynchronously.

Next steps

Start with the validation piece that catches the most leaks: read purged walk-forward validation, explained simply, then try an ML run on one of your own strategies.

Frequently asked questions

Can machine learning predict crypto prices?

Machine learning can sometimes find weak, conditional edges, such as which of a strategy's signals tend to work, but it does not reliably predict prices. Most apparent ML edges in backtests come from leakage or overfitting, which is why validation matters more than model choice.

What is the most common cause of overfitting in ML trading models?

Information leakage through validation: shuffled k-fold splits, overlapping labels that straddle the train/test boundary, and features that peek at data not available at decision time. Testing many variants and reporting the best one is the second big cause.

Does Trigr tune hyperparameters when it optimizes a strategy with ML?

No. Trigr compares LightGBM, random forest and XGBoost, each with fixed, disclosed parameters, and selects the family by anchored, purged walk-forward out-of-sample results. Hyperparameters are not optimized.

Which model is best for a machine learning trading strategy?

For tabular market features, gradient-boosted trees and random forests are sensible defaults because they handle non-linear interactions with little preprocessing. The choice of model matters far less than point-in-time features, honest labels and purged validation.

Is a good out-of-sample Sharpe from an ML run enough to trade it?

No. Check the Deflated Sharpe Ratio, the probability of backtest overfitting and any leak flags, then forward-test on paper. Repeating optimization runs on the same data also erodes the meaning of any out-of-sample number.

Put the idea to an honest test.

Describe a strategy in plain English or from your own AI assistant, backtest it on point-in-time data, and forward-test it on paper before any real money is involved.