The Most Valuable Thing We Built Wasn't a Signal

In early 2025, we had a signal that looked like a breakthrough.

Signal 1 was returning 95% accuracy on the BTC directional prediction problem we had been working on for months. The work behind it was real — hundreds of thousands of five-second bars, a staged discovery process, careful feature engineering. The result was internally celebrated.

We built a leak detector.

The leak detector found a 60-second look-ahead bug threaded through the momentum features. Future data was contaminating the training set. The model had learned to predict the future because it had already seen it.

After the bug was removed, the accuracy collapsed to 56%.

We retracted Signal 1.

That decision — to build the infrastructure that could kill our own best result, and then to act on what it found — is the most important thing that happened in eighteen months of research. Not because it salvaged the project. Because it proved the methodology was worth trusting.

What a Leak Detector Is

In quantitative research, a look-ahead leak is a form of contamination where information from the future — even milliseconds of it — is inadvertently available to a model during training.

Markets are unforgiving about this. If your feature calculation includes data that would not have been available at decision time, the model learns patterns that cannot exist in production. The backtest looks extraordinary. The live system fails immediately, usually without obvious explanation.

The honest version of this problem: most research programmes never find their own leaks. The system looks good. The backtest holds. The team moves forward. The edge collapses in production and the most common diagnosis is “the market changed.”

The market didn’t change. The look-ahead was there from the beginning.

A leak detector is infrastructure built specifically to catch this. It replays the dataset chronologically, verifies that every feature observation used a computation window that was closed at decision time, and flags any bar where future data was accessible. Ours mapped exactly which days were clean versus artifact-prone — a fidelity audit that became a standing constraint on which data could enter any subsequent research.

Building this infrastructure is a choice. It takes significant engineering time. It will find things you don’t want to find. The signal that took months to develop may not survive.

Why We Built It Anyway

The research programme was never about Signal 1 specifically. It was about building a machine that could find durable edges in high-frequency market microstructure — and more importantly, a machine that could distinguish durable edges from statistical artefacts.

The shuffled-null control was the other half of this discipline. Every candidate signal was evaluated not just against the data but against a version of the data where the target labels had been randomly permuted. If a signal that should have no predictive relationship with outcomes still looked good on the shuffled target, it was a sign of overfitting, not discovery. A signal only passed if its in-sample results substantially outperformed the shuffled null.

The combination — leak detection plus null-control testing — means the discovery process is designed to be sceptical of its own outputs. The filter is set up to fail most of what it finds, deliberately. That is not a sign of a failing research programme. It is a sign of a programme that will eventually produce something worth trusting.

The Institutional Bot Reconstruction

The underlying research question was this: on BTC/USDT, large institutional players run VWAP-style execution programmes — aggressive taker buying into pre-placed sell walls on a repeating cycle — and if you can see the programme forming, you can trade with or against it.

The reconstruction machinery — raw trade tape clustering, detection of repeated same-direction pushes and large sweeps, order book context around each detected programme — was built to reverse-engineer this behaviour from first principles.

The finding held. Approximately 100 to 135 institutional execution episodes are detectable per day. Their impact on price builds measurably with participation share. The impact partially reverts after the programme completes. This pattern held across multiple data eras against control comparisons.

We rebuilt the same bot independently in two separate codebases, arriving at the same structural conclusions without cross-referencing the implementations. The consistency was the validation.

These are what we called “laws of the market” — multiply-replicated regularities that survive across datasets, across implementations, and across time. The Exhaustion Law: visible institutional execution marks the end of moves, not the continuation. Leg continuation: the structure of a move predicts its continuation probability. Sensors have habitats: certain signals are only valid within specific market conditions.

Laws of this kind are not found by running a regression. They are found by building the infrastructure to fail most of what you test and keeping only what survives honest scrutiny.

The Scale of What Was Built

Eighteen months. Approximately three thousand Python files across three codebases. More than four hundred thousand lines of code. Around forty trained models — GRU specialists, Mamba state-space models, a dual-stream CNN, an eight-model voting ensemble with a meta-routing layer. A GPU training pipeline confirmed to RunPod H100 infrastructure.

Three independent foundation model forecasters — Amazon Chronos-2, Google TimesFM, Salesforce Moirai — running in agreement on live data, firing signal packs only when all three concur. A crash-hardened twenty-service live engine that had its C-level crashes debugged, its Windows-native process isolation hardened, and its faulthandler instrumented.

The honest ledger alongside all of this: the strongest validated candidate from the research is real-shaped — a confirmed directional regularity with detectable edge per trade — but not yet at the statistical power required to claim a proven live trading advantage. The wall between a promising research finding and a deployable edge is statistical power, and statistical power requires time and sample size.

The hunt continues. That is the point.

Why This Belongs on a TVS Article

The methods described in the TVS market intelligence system — the staged discovery process, the chronological freeze, the branch discriminator audit described in the correlation engine articles — did not come from a textbook. They came from eighteen months of building a system that killed more than it kept.

The same discipline is now applied to search ranking data. Two hundred and forty-nine businesses in eighteen KZN industries, adding one observation per business per day. The dataset is not yet deep enough to run the full discovery protocol at the reliability the financial market research eventually reached. Markets generate hundreds of thousands of events per day; search rankings generate one per business per day. Statistical convergence takes longer.

But the infrastructure — the commitment to building data the right way, the refusal to claim what hasn’t been validated, the architecture that can honestly retract a finding — is in place.

The market intelligence system will eventually produce findings about KZN search ranking that are as rigorously validated as the institutional bot reconstruction. The methodology is the same. The domain is different. The discipline is identical.

What we said about Signal 1 is what we will say about any TVS finding that doesn’t survive honest testing: we retract it. Most shops never run the test. We built the infrastructure specifically so we have to.

That is not a limitation. It is what makes the results worth trusting when they do survive.

---

The correlation discovery methodology is documented at The Correlation Engine and After the Discovery. The specialist committee architecture the research produced is at The Specialist Committee. The evolutionary signal search method is at How We Search for Signals We Haven’t Imagined Yet. The institutional bot reconstruction is at Market Physics: Why We Model Behaviour Before We Model Price. What the production system’s crash history looks like after eighteen months of live operation is at The Crash Diary. The TVS market intelligence dataset is described at What 249 KZN Businesses Reveal.