七个交易想法，诚实测试：记分牌

Engineering field note. Over four weeks we were handed seven trading ideas — most of them from quantitative researcher Vladislav Freidyn. We built each one, tested it against the same fail-closed protocol, and wrote down what happened. None of them produced a proven edge. This note is the scoreboard, because the interesting part is not the verdicts — it is that every failure left a permanent instrument behind, and the instruments are now what makes the next test cheap.

The protocol every idea meets

An idea does not get a verdict from one backtest. It walks a ladder of gates, each of which has killed at least one of the seven ideas below. The order matters: the cheap, brutal gates come first.

The scoreboard

Seven experiments, four weeks, one protocol. The last column is the number that ended each one.

Three failures worth studying

1. The sparsity illusion

The first campaign produced a headline of +14.52% out of sample on one cell. It was worthless, and the reason is the most common trap in strategy research: the return was a median over coins, and almost every coin contributing to it had one to three trades. Measured properly — how many coins were profitable out of sample with a real sample size — the same cell scored 0 of 19.

The same campaign taught a second lesson within hours. The star cell of the first wave — 13 of 20 coins profitable on the fast frame — evaporated when we replaced 30 days of history with several years. On deep data every setup on that frame was negative, from −9.2% to −41.5%. The fix for a weak result is more history, not more tuning.

2. The information-free twin

A filter that skips trades will almost always improve your win rate, because it removes trades. That is not edge, it is arithmetic. The only way to tell the difference is to build a twin filter that thins the sample by the same amount using no information at all — a mechanical every-other-signal veto — and compare against that.

We did, on the market-wide veto idea. On the daily frame the real filter looked excellent. The information-free twin looked slightly better. On the four-hour frame the twin actively hurt while the real filter helped. Same idea, same code, opposite conclusions — and without the twin we would have shipped the daily version.

3. When the model loses to a constant

The most ambitious idea was to let gradient boosting read the market state — dollar index, funding, open interest, volume delta, structure breaks, correlation, sentiment — and decide which strategy family fits right now. We built it as a candidate ranker inside the existing gates rather than as an oracle, and ran it across the universe.

0 of 169 symbols showed predictive lift. The model scored worse than a constant prediction in every one of eight market slices and all four folds. Adding a sentiment feature made it worse, not better. But the same run produced the single most useful output of the whole series: the configuration that simply declined to trade when expected value was negative took zero trades, returned zero, and beat every model. No Trade is a first-class answer, and it won.

What the failures built

This is the part that compounds. Each experiment left an instrument in the pipeline, and the pipeline is now strict enough that the last idea was falsified in a single day instead of a week.

What the last experiment looked like with all of it switched on

The most recent idea — break a volatility channel while the asset and the majors both turn — met the finished pipeline. It is a useful demonstration of how fast a fair test now runs, and of how many different ways a strategy can be found wanting.

• Search: 913 parameter sets across three timeframes, presented as a trade-off frontier rather than one winner. Best in-sample result: +12.2%.
• Holdout: of the 30 strongest candidates, one survived every gate — and it beat simply holding the same short position by 0.01 percentage points on 14 trades.
• Breadth: transferred unchanged to 29 other coins it passed on 1, where chance alone predicts 0.55. Re-searched separately per coin it passed on 1 of 25, where chance predicts 1.25 — below luck.
• Cross-asset: on 50 non-crypto instruments it barely fired at all — 41 produced no trades, because a percentage-width channel is a crypto-volatility construct. Not applicable rather than unprofitable, and worth saying precisely.
• Shelf life: positive only in the first stretch after each re-fit, worth about 65 cents per ten days on a 500 dollar book. Nothing to decay.

Two things worth knowing about testing someone else's strategy

Porting a chart script into a testable engine is where quiet errors live. Two examples from this series, both of which changed the answer:

• A control that does nothing. In the latest script, the moving-average block the trader had been tuning for weeks — timeframe, period, shift — is only drawn on the chart. It never enters the entry condition. Those inputs cannot change a single trade, and no amount of optimising them would have mattered.
• A stop that is not armed. The same script computes its stop from the average entry price at a moment when the position is still empty, so the platform silently discards the bracket. The first position therefore runs unprotected until a second entry arms one. Modelling it the intuitive way instead of the real way flipped the backtest from −2.00 to +15.90 dollars — the sign, not the magnitude.

Neither is a criticism of the trader. Both are the reason a strategy has to be re-implemented and diffed against the original report rather than trusted, and the reason we publish the diff.

The eighth idea, and where the method came from

The series has an eighth entry that is not on the scoreboard, because it is not a strategy. In July 2026 Vladislav published Search Edge: A Market Regime-Aware Decision Engine for Algorithmic Crypto Trading — an architecture rather than a rule set, and worth reading in his own words rather than our summary.

We could not test it, because there is nothing in it to execute. What we could do was check its proposals against what we had already measured, and take the parts we were missing:

• Its core we had already refuted. The gradient-boosting selector is the same hypothesis as the machine-learning experiment above, and it failed on 169 symbols. That is not a rebuttal of the architecture — it is the result of testing it, which is the more useful thing to have.
• Its validation section we adopted. Purged and embargoed cross-validation was the one genuine gap in our pipeline, and it is now implemented, including a hole sized from measured holding time and a self-check that refuses to publish a leaky number. So are risk-adjusted axes on the trade-off frontier, and the shelf-life measurement.
• Its strongest point we had independently confirmed. No Trade as a first-class decision is the one thing in the whole series that measurably won.

What would change a verdict

None of these ideas is closed forever. Each has a stated condition that would reopen it, and the conditions are deliberately falsifiable:

• the macro layer becomes testable when history covers several macro regimes, not one;
• the market-wide veto becomes shippable when it holds through a regime it was not tuned in;
• the averaging ladder becomes interesting where its result is broad across the grid rather than a right tail;
• the channel break becomes interesting if it survives on instruments it was never fitted to;
• the machine-learning core becomes interesting with a different target definition, not with more features.

Until one of those happens, the honest output is the scoreboard above and the instruments below it. We would rather publish seven negatives and a working test bench than one curve that only exists in the past.