Statistical Significance of a Track Record: How Many Trades It Takes to Tell Skill From Luck

A track record is statistically significant when its average profit per trade is large compared with how much individual trades vary, given how many trades there are. The standard test is a t-statistic, which for a list of independent trades equals the per-trade Sharpe ratio multiplied by the square root of the number of trades. For the edges signal providers usually claim, that means several hundred to several thousand trades before skill can be told apart from luck.

In short: a 55% win rate on trades that win and lose the same amount needs about 400 trades to reach a t-statistic of 2, and about 900 to reach 3. Measured in calendar time with monthly returns, an annual Sharpe ratio of 1.0 needs roughly four years of data to clear t = 2 (Andrew Lo’s standard error formula, Financial Analysts Journal, 2002). Serial correlation, fat tails and negative skew all push the requirement up, and the number of providers you could have picked from pushes it up again. Three months of screenshots cannot establish skill at all, whatever they show.

This article assumes you know what a win rate and a risk-reward ratio are (see why win rate is misleading). It explains a statistical method and shows the arithmetic. It does not rate or recommend any provider.

A common misconception: “the results speak for themselves”

The typical pitch shows a strong recent run: “74 wins from the last 100 signals” or “+1,850 pips this quarter”. The implied claim is that a result this good cannot be an accident. Whether that is true depends on three things the pitch never tells you:

How variable the trades are. A record built from +20 pip wins and -200 pip losses can show a high win rate with no edge at all.
How many trades there are. Small samples produce extreme results often. That is just arithmetic, with nothing suspicious about it.
How many other records you didn’t see. If a thousand people (or one person running many accounts) start trading, the best of them will look skilled even if none is.

Statistics gives you a way to put numbers on the first two, and a separate set of tools (covered in overfitting and data snooping) for the third.

The t-statistic of a track record

Express each trade as a multiple of the amount risked, called R. A trade that hits a stop is -1R; one that hits a target twice as far away is +2R. This removes the effect of changing position sizes and lets you pool trades. (Converting pips to R is covered in pips, lots and signal math.)

For n trades with average R of m and standard deviation s:

t = m / (s / sqrt(n)) = (m / s) x sqrt(n)

The ratio m / s is the per-trade Sharpe ratio. Under the null hypothesis that the true average is zero, t is approximately standard normal for large n. A t of 2 corresponds to a one-sided p-value of about 2.3%; a t of 3, about 0.13%.

Rearranging gives the number of trades needed to reach a target t:

n = (t / (m / s))^2

The required sample grows with the square of the inverse edge. Halve the edge and you need four times as many trades.

Worked example 1: symmetric trades (win +1R, lose -1R)

With win probability p, the expectancy is m = 2p - 1 and the standard deviation is sqrt(1 - m^2). All figures below are computed in Python.

Win rate	Expectancy per trade	Per-trade Sharpe	Trades for t = 2	Trades for t = 3
51%	0.02R	0.020	9,996	22,491
52%	0.04R	0.040	2,496	5,616
53%	0.06R	0.060	1,108	2,491
55%	0.10R	0.101	396	891
58%	0.16R	0.162	153	343
60%	0.20R	0.204	97	217

Worked example 2: trades with a 1:2 risk-reward (win +2R, lose -1R)

Here m = 3p - 1. The payoff is more variable, so the same expectancy needs more trades than in the symmetric case.

Win rate	Expectancy per trade	Standard deviation	Per-trade Sharpe	Trades for t = 2	Trades for t = 3
35%	0.05R	1.43R	0.035	3,277	7,372
36%	0.08R	1.44R	0.056	1,296	2,916
38%	0.14R	1.46R	0.096	433	974
40%	0.20R	1.47R	0.136	216	486
45%	0.35R	1.49R	0.235	73	164

An expectancy of 0.10R to 0.20R per trade, before costs, would be a strong result for a discretionary or rule-based forex strategy. Even at that level, the honest sample is several hundred trades. A provider sending three signals a week produces about 150 trades a year.

Worked example 3: what a 55% record over 200 trades actually shows

Suppose a provider shows 200 symmetric trades with a 55% win rate.

Trades	t-statistic	One-sided p-value	95% interval for the true win rate
50	0.71	0.239	41.2% to 68.8%
100	1.01	0.157	45.2% to 64.8%
200	1.42	0.078	48.1% to 61.9%
400	2.01	0.022	50.1% to 59.9%
1,000	3.18	0.0007	51.9% to 58.1%

At 200 trades, the data are consistent with a true win rate anywhere from just under a coin flip to almost 62%. The record is not evidence against skill, but it is not evidence for it either. It only becomes informative somewhere around 400 trades, and only if those trades are independent and the record is complete.

The test also misses real skill

The trade counts above say when a genuine edge is expected to reach t = 2. Half the time it will fall short. Take a provider whose true win rate really is 55% on symmetric trades, and ask how often its record would clear t = 2 (the normal approximation, checked against exact binomial probabilities):

Trades in the record	Expected t-statistic	Chance a genuinely skilled 55% record clears t = 2
100	1.01	16%
200	1.42	28%
400	2.01	50%
800	2.84	80%
1,600	4.02	98%

This is the power of the test, and it is low for any record of realistic length. It cuts both ways for someone choosing between providers. A record that fails the test is not proof of no skill, because most skilled records of a few hundred trades fail it too. But a record that passes is not proof of skill either, unless you also know how many unskilled records were in the pool it came from, which is the subject of the section on luck at scale below. With low power and a large pool, most of the records that pass can be lucky ones. Statisticians call this the false discovery problem, and it is why a single impressive record carries so little information on its own.

Moving to returns: the Sharpe ratio standard error

Many providers and copy-trading platforms report monthly returns rather than trades. The relevant statistic is then the Sharpe ratio of monthly returns, and the relevant paper is Andrew W. Lo, “The Statistics of Sharpe Ratios” (Financial Analysts Journal, vol. 58, no. 4, 2002, pp. 36-52).

Lo’s starting point is that the Sharpe ratio is built from two estimated quantities, the mean and the standard deviation, so it carries estimation error like any other statistic. Under the assumption that returns are independently and identically distributed (IID), he derives the large-sample standard error:

SE(SR) = sqrt( (1 + SR^2 / 2) / T )

where SR is the per-period Sharpe ratio and T is the number of periods. Lo’s Table 1 gives, for example, a standard error of 0.188 for a true Sharpe ratio of 1.50 with 60 observations, and 0.303 for a Sharpe ratio of 3.00 with the same sample. Our implementation reproduces both figures.

The SR^2 / 2 term is the contribution of estimating the volatility. Lo shows that for a small Sharpe ratio such as 0.25, 97.0% of the estimator’s variance comes from the mean; for a Sharpe ratio of 2.00, only 33.3% does. For monthly Sharpe ratios of typical size, the mean dominates, which is why the t-statistic and the Sharpe ratio test give nearly the same answer.

Worked example 4: confidence intervals for an annual Sharpe ratio

Take an annualized Sharpe ratio, convert it to monthly by dividing by sqrt(12), apply Lo’s IID formula, and scale back up. (The IID assumption matters here; see the next section.)

Annual Sharpe	Years of monthly data	Standard error (annual)	95% confidence interval	t-statistic
0.5	1	1.01	-1.47 to 2.47	0.50
0.5	5	0.45	-0.38 to 1.38	1.11
0.5	10	0.32	-0.12 to 1.12	1.57
1.0	1	1.02	-1.00 to 3.00	0.98
1.0	3	0.59	-0.15 to 2.15	1.70
1.0	5	0.46	0.11 to 1.89	2.19
1.0	10	0.32	0.37 to 1.63	3.10
2.0	1	1.08	-0.12 to 4.12	1.85
2.0	3	0.62	0.78 to 3.22	3.21

One year of monthly returns tells you almost nothing about a Sharpe ratio. The standard error is about 1.0 regardless of the true value, so the interval spans roughly four Sharpe units.

Worked example 5: years needed to reach significance

Solving Lo’s formula for T at a target t-statistic:

Annual Sharpe	Years for t = 2	Years for t = 3
0.5	16.2	36.4
0.75	7.3	16.4
1.0	4.2	9.4
1.5	1.9	4.4
2.0	1.2	2.6
3.0	0.6	1.4

This table cuts both ways. A genuinely excellent strategy (Sharpe 2 or more) can show itself within a year or two. But a claimed Sharpe ratio of 2 to 3 from a retail signal service is itself an extraordinary claim, and the shorter the record, the more likely it is that the high Sharpe ratio is the product of a small, lucky sample rather than a stable edge.

Serial correlation: why annualizing by sqrt(12) can mislead

The convention of multiplying a monthly Sharpe ratio by sqrt(12) is correct only for IID returns. Lo derives the general conversion for returns that are correlated over time. For a first-order autocorrelation rho, his Table 2 gives these scale factors for monthly to annual:

Monthly first-order autocorrelation	Scale factor (monthly to annual Sharpe)
-20%	4.17
0% (IID)	3.46
+20%	2.88
+40%	2.36 (our calculation from Lo’s formula)

Positive autocorrelation means losses and gains cluster, so annual volatility is higher than sqrt(12) times monthly volatility, and the naive conversion overstates the annual Sharpe ratio.

Lo’s empirical example shows how large the effect can be. Among 12 hedge funds, the IID estimator gave a “Mortgage-backed securities” fund an annual Sharpe ratio of 4.03; accounting for serial correlation cut it to 2.44, an overstatement of 65%. A “Convertible/option arbitrage” fund fell from 4.35 to 2.99. One fund with negative autocorrelation went the other way, from 3.25 to 3.83, and the rank ordering of the 12 funds changed substantially. The ten large mutual funds in the same study had little serial correlation, and the correction barely mattered for them.

Why this matters for signal records: returns become serially correlated when trades overlap, when positions are held across many reporting periods, when a strategy averages down, or when losing positions are left open and not marked to market. Grid and averaging-down systems (see martingale and grid strategies explained) produce exactly this kind of smoothed, autocorrelated equity curve. The smoother the curve looks, the more you should ask why.

Fat tails and skew: the Probabilistic Sharpe Ratio

The t-test and Lo’s IID formula both lean on approximately normal returns. Trading returns are often skewed. A strategy that wins small amounts most of the time and occasionally loses a lot has strong negative skew and high kurtosis, and its Sharpe ratio is less reliable than a normal-distribution calculation suggests.

David Bailey and Marcos López de Prado proposed the Probabilistic Sharpe Ratio (PSR) in “The Sharpe Ratio Efficient Frontier” (Journal of Risk, vol. 15, no. 2, 2012). It gives the probability that the true Sharpe ratio exceeds a benchmark SR*, adjusting for sample length, skewness (g3) and kurtosis (g4):

PSR = Phi( (SR - SR*) x sqrt(T - 1) / sqrt(1 - g3 x SR + (g4 - 1) / 4 x SR^2) )

Here Phi is the standard normal cumulative distribution and SR is the per-period (here per-trade) Sharpe ratio. Inverting it gives the minimum track record length (MinTRL): the number of observations needed for PSR to reach a chosen confidence level. Bailey and López de Prado’s later paper, “The Deflated Sharpe Ratio” (Journal of Portfolio Management, 2014), uses the same formula; we confirmed our implementation by reproducing that paper’s published numerical example exactly.

Worked example 6: minimum trades at 95% confidence for three payoff shapes

Payoff profile	Expectancy	Per-trade Sharpe	Skew	Kurtosis	MinTRL (actual moments)	MinTRL (if assumed normal)
55% win, +1R / -1R	0.100R	0.101	-0.20	1.04	274 trades	270 trades
30% win, +3R / -1R	0.200R	0.109	+0.87	1.76	207 trades	230 trades
92% win, +0.3R / -3R	0.036R	0.040	-3.10	10.59	1,889 trades	1,676 trades

Kurtosis here is the raw (non-excess) value, so a normal distribution has 3. The third row is the shape many high win rate systems produce: a 92% win rate, with every loss ten times the size of a typical win. Its true edge is small, its skew is strongly negative, and it needs close to 1,900 trades to establish that edge at 95% confidence. A provider advertising a 92% win rate over 150 trades has shown you a payoff profile, not a demonstrated edge.

The positively skewed trend profile in the second row needs fewer trades than the normal calculation suggests. Skew is not bad in itself; negative skew paired with small samples is what hides risk.

Luck at scale: the best of many records

Everything above tests one record chosen in advance. That is rarely how records reach you. Halbert White opens “A Reality Check for Data Snooping” (Econometrica, vol. 68, no. 5, 2000) by likening data snooping to “running the newsletter scam”: mail a free market prediction to a large list, telling half “up” and half “down”; keep mailing only those who received the correct call; after several months, “there can still be a rather large group who have received perfect predictions, and who might pay for such ‘good’ forecasts.”

You don’t need fraud to get the same effect. Simulate 1,000 providers with no edge at all, each trading 100 symmetric +1R / -1R signals:

Win rate reached by a zero-edge provider	Probability	Expected number out of 1,000
55% or better	18.4%	184
58% or better	6.7%	67
60% or better	2.8%	28
62% or better	1.1%	11
65% or better	0.18%	2

In 20,000 simulated batches of 1,000 providers, the best record had a median win rate of 66% (5th to 95th percentile: 64% to 69%). If you browse a leaderboard of a thousand signal channels and pick the top one, a 66% win rate over 100 trades is roughly what zero skill delivers. This is the mechanism behind survivorship bias in signal providers: the records you are shown are selected precisely because they look good.

Campbell Harvey and Yan Liu make the general point in “Backtesting” (Journal of Portfolio Management, vol. 42, no. 1, 2015): “Although a t-ratio of 3.0 is highly significant in a single test, it may not be if we take multiple tests into account.” How to adjust for the number of records searched is the subject of the companion article on overfitting.

What can go wrong with the test itself

Trades are not independent. Five signals on EUR/USD, GBP/USD and EUR/JPY opened during the same US dollar move are closer to one bet than five. Counting them separately inflates n and the t-statistic. Group trades that overlap in time and share a common driver, or test on daily account returns instead.
Position sizes change. Pip totals mix large and small positions. Test R multiples or percentage returns on a fixed account, never raw pips.
The record is incomplete. Deleted signals, “breakeven” trades that were really losses, and closed channels all bias the mean upward. No statistic can fix missing data. Start with how to verify a signal track record.
The start date was chosen afterwards. A record that begins right after a bad run is a selected sample.
The past is not the future. Significance tells you the historical mean was probably not zero. It does not say the edge will persist after costs, slippage on copied trades, or changes in market regime.
Open losses are not counted. A strategy that holds losing trades open has unrealized losses that a closed-trade record omits. Equity-based returns capture them; closed-trade lists don’t.

A practical reading of any record

Convert every trade to R or to percentage return on a fixed account.
Compute the mean, standard deviation, skewness and kurtosis.
Compute the t-statistic and the minimum track record length for the observed moments.
If returns are reported monthly, check the first-order autocorrelation before annualizing.
Ask how many other strategies, accounts or channels the same provider runs or has run, and treat the record as the best of that many.

If the record fails step 3 on its own terms, steps 4 and 5 only make it worse.

The short version

The t-statistic of a track record equals the per-trade Sharpe ratio times the square root of the number of trades, so the trades needed grow with the square of the inverse edge: about 400 for a 55% symmetric record to reach t = 2, and about 1,100 for 53%. Lo’s 2002 standard error formula shows that one year of monthly returns leaves an annual Sharpe ratio uncertain by about plus or minus 2, and that serial correlation can overstate annualized Sharpe ratios by 65% in real fund data. Negative skew, typical of high win rate systems, raises the minimum track record length further: our 92% win rate example needs about 1,900 trades. And when the record was picked from many, even a 66% win rate over 100 trades is what pure luck produces.