Psychology spent the better part of a decade discovering that a large share of its most cited findings didn't replicate. The studies were real, the statistics were real, the p-values cleared the bar — and a meaningful fraction of the results still evaporated the moment an independent lab tried to reproduce them. The field's response to that reckoning is the useful part of the story: pre-registration. Commit to your hypothesis, your method, and what would count as failure, in writing, before you collect a single data point. Then run the study. You don't get to discover the interesting pattern afterward and call it the thing you were testing for all along.
Markets have the identical problem, at a larger scale, with far less institutional pressure to fix it. Run enough variations of a trading idea against enough historical data, and something will look like an edge purely by chance — not because the market rewards it, but because with enough parameters and enough retries, noise eventually arranges itself into a pattern that resembles signal. The idea that gets published, or funded, or traded live isn't the one that was right. It's the one that happened to look right on the specific slice of history someone tested against, after however many attempts it took to find it.
The Question That Changes Everything
The honest question to ask about any trading idea isn't "did it work." It's "would I have believed in this before I saw the result." Almost every idea passes the first test, because the test is run after the fact, by someone who already knows which version worked. Almost no idea survives the second test, because the second test requires you to have written down, in advance, exactly what would make you reject it — and then to actually reject it when that happens, instead of quietly adjusting the definition of success until the result fits.
"A result you found by trying is data. A result you predicted before you looked is evidence. The two are treated as identical by almost everyone, and they are not remotely the same thing."
Why the Rejection Matters More Than the Promotion
The part of scientific pre-registration that markets skip entirely isn't the commitment to a hypothesis — plenty of research processes have a hypothesis. It's the commitment to publish the failure. A pre-registered study that fails still gets written up, because the negative result is itself information: this specific, well-specified idea did not hold. A trading idea that fails its pre-committed test almost never gets written up anywhere. It just gets quietly abandoned, and the next idea gets tried, and nobody keeps a record of how many were rejected before one was promoted. Without that record, you have no way to tell a genuine edge from the seventh idea that happened to clear a threshold after six others didn't.
What a Research Ledger Actually Records
This is the discipline underneath Obsidian's research process, and it's why the word for what we keep is a ledger, not a track record of what worked. Every idea enters with its hypothesis and its rejection criteria fixed in advance. Most of them get rejected — that's not a failure of the process, it's the process functioning exactly as intended, the same way a pre-registered study that comes back negative is a successful piece of science, not a wasted one. An idea gets promoted only after it survives a test it could have failed, specified before anyone knew whether it would.
The alternative — running an idea, liking what you see, and then constructing the story about why it should have worked — produces something that looks identical to research from the outside. It has charts. It has a thesis. It sounds rigorous when you say it out loud. The only thing missing is the one property that actually matters: it was never at risk of being wrong, because the definition of right was written after the answer was already known.
The Uncomfortable Part
A process built this way rejects far more ideas than it promotes, and that ratio doesn't improve much no matter how good the research gets — because the point of the discipline isn't to find more edges, it's to stop false ones from surviving on the strength of a good story. That's an uncomfortable trade for anyone measuring output by how many ideas make it through. It's the only trade that produces a ledger you can actually trust six months later, when the market conditions that flattered a lucky pattern have quietly moved on and only the ideas that survived a real pre-committed test are still standing.
That's the whole argument, reduced to one sentence: if you didn't write down how it could fail before you looked at whether it did, you don't have an edge. You have a story, and the market is exceptionally good at making stories feel true right up until the moment it isn't paying you for one anymore.