Confirmation Bias, Measured

Three ordinary indicators at four ordinary lookbacks make twelve configurations, and all twelve are defined on 39 of this module’s sixty closes. On 20 of those 39 bars at least one of them reads bullish and at least one reads bearish, so half the time the chart will hold whichever answer you brought to it. It is more lopsided than that in one direction: 34 of the 39 bars carry a bullish reading and only 25 carry a bearish one, which is why a bull almost always finds company and a bear sometimes has to wait. The twelve are not twelve opinions. Their vote series correlate at 0.5455 on average, which is 1.71 independent opinions by lesson 48’s count, and the thing they actually disagree about is not the indicator but the lookback: change the family and keep the lookback and they agree 91 per cent of the time; keep the family and change the lookback and they agree 76. None of the twelve beats doing nothing. The best of them is right about the next bar 59.3 per cent of the time and the tape closes higher on 60.5 per cent of the same bars.

Prerequisites: Lesson 48, which showed that three indicators on one price series correlate at 0.8348 and are worth 1.12 readings, and whose formula produces the 1.71 above, lesson 50, which priced separating two hit rates near these at 609 observations and is the reason nothing here is claimed from thirteen bars, and lesson 51, whose oscillator is one of the three families below and whose fifty line is the threshold it is read against.

And none of them is worth anything anyway

The obvious defence is that the disagreement is a feature: the configurations that are right will separate from the ones that are wrong, and the reader’s job is to find them. So measure that. For each configuration, take every bar on which it reads bullish and ask whether the next close is higher. Then ask the same question of every bar in the window, which is the base rate lesson 50 insisted on. Bar 60 has no next close, so it is out of every column including that one.

Configuration	Bullish bars	Next close higher	Rate
Average 5	25	13	52.0%
Average 10	27	15	55.6%
Average 15	27	16	59.3%
Average 21	26	15	57.7%
Oscillator 5	23	12	52.2%
Oscillator 10	27	16	59.3%
Oscillator 15	26	15	57.7%
Oscillator 21	22	13	59.1%
Momentum 5	23	12	52.2%
Momentum 10	27	16	59.3%
Momentum 15	26	15	57.7%
Momentum 21	22	13	59.1%
Every bar in the window	38	23	60.5%

Read the last row first. Twelve configurations, and the best of them is 59.3 per cent against a tape that closes higher on 60.5 per cent of the same bars with no reading consulted at all. Not one of the twelve clears the base rate. The bullish column runs from 22 bars to 27 because the twelve do not fire on the same bars, and the two exact ties take one more away from the five-bar oscillator and the five-bar momentum. The rates land in a band seven points wide with the base rate above every one of them.

Then the sorting itself. Split the 38 bars by what the twelve did rather than by what any one of them said. On the 13 bars where all twelve agreed bullish, the next close was higher on 5, which is 38.5 per cent. On the 19 bars where they split, it was higher on 14, which is 73.7. On the 5 bars where all twelve agreed bearish, higher on 3.

Agreement was the worst thing on the chart and disagreement was the best. That is backwards from every way the twelve are ever used, and it has an explanation that costs nothing to believe: twelve configurations agree when the last month has moved one way without interruption, and a run that has already gone one way without interruption is the state in which the next bar is least likely to extend it. The unanimous readings are not a consensus. They are a description of what already happened, arriving late, all twelve at once, because they are looking at the same thing through the same window.

Thirteen bars and nineteen bars are not enough to sell that finding to anybody, and the concessions below say so in the terms lesson 50 set. What they are enough for is the smaller claim, which is the one this lesson is making: the disagreement a reader resolves in their own favour is not a disagreement between better and worse readings. There is no better reading in the table.

What this does not settle

That the thirteen-against-nineteen split is real. It is not established and it cannot be from these numbers. Thirteen unanimous bars returning 38.5 per cent against a base rate of 60.5 is a gap of twenty-two points, and the standard error on the 38.5 alone is thirteen and a half points while the standard error on the difference is fifteen and a half, and lesson 50 priced telling apart two rates closer than these at 609 observations in each condition. The direction is worth naming and the magnitude is worth nothing. Everything in this lesson that survives is in the columns that count configurations, not in the ones that count outcomes.

That twelve is the right number, or these twelve the right twelve. Three families and four lookbacks was a choice made before the computation and it drives the correlation directly. Add a fourth family that is genuinely built differently — volume, or the spread between two instruments — and the 0.5455 falls and the 1.71 rises. What does not change is the ordering: any two configurations sharing a lookback will agree more than any two sharing a family, because a lookback is a statement about which bars are being described and a family is a statement about the arithmetic applied to them.

That the two exact ties are a detail. They are the reason two columns in the worked table hold 23 and 22 bullish bars where their neighbours hold 26 and 27, and a reader reproducing this who counts a tie as bullish will get different numbers in four cells. The convention is stated because it has to be, not because it is interesting, and it is a reminder that the same silence sits inside every indicator: your platform has an answer for the tie and it has never told you what it is.

That direction is what the twelve actually say. Every configuration on this page was reduced to a sign, and readers do not use them that way: they read the size of the gap, the slope of the line, the distance from the level. A configuration reading 51.2 against 50 and one reading 75.0 against 50 both counted as one bullish vote at bar 29, which is exactly the reduction that made the disagreement countable and exactly the information a careful reader would say was thrown away.

That any of this is a measurement of confirmation bias. It is a measurement of what is available to be found, which is a fact about the chart. Whether a particular reader reaches for the configuration that agrees with them is a fact about the reader, and this page has no access to it. The finding here is weaker and harder to escape: on half the bars in this window, being honest is not enough, because there is an honest reading on both sides.

And the base rate the whole worked table is measured against is generous, in a way that flatters this lesson’s own conclusion. All twelve are defined only from bar 22, and bars 22 to 60 are the stretch of this series that trends: the tape closes higher on 60.5 per cent of them against 42.9 per cent over the first twenty-one bars and 54.2 per cent across all fifty-nine. Measure the twelve against the whole series’ 54.2 instead and nine of them clear it, five by more than four points. That is the wrong comparison — a base rate has to come from the same bars as the thing it is judging — but a reader who suspects the headline was helped by where the window fell is right to suspect it, and the honest version of the sentence is that none of the twelve beats the tape on the bars where all twelve exist.

Problems

1. Count how many opinions are on your screen. Take whatever indicators you have loaded, write down what each one has to be for you to call it bullish, and record that as a plus or a minus on every bar of the last sixty. Then count the bars where they do not all say the same thing. If that count is near half, the screen is not confirming anything; it is offering. Ten minutes, and you will not need the rest of this lesson to know what to do about it.
2. Separate the lookback from the indicator. Take three of your indicators and run each at two lookbacks, one about a week and one about a month. Six columns. Now count agreement two ways: same lookback across different indicators, and same indicator across different lookbacks. On this lesson’s series those come out at 91 and 76 per cent. If yours come out anything like that, the three indicators were never the diversification — the two lookbacks were, and you have two opinions rather than six. Half an hour.
3. Put every column against the base rate on its own bars. For each configuration, take the bars where it reads bullish and record whether the next close was higher. Then take every bar in the same window and record the same thing. Twelve rates and one base rate, and the only question is how many of the twelve clear it. Do it for five bars ahead as well as one, because the answer moves. Keep going until each column holds a hundred bullish bars rather than twenty-five, which is the only way any of it becomes a measurement. An evening, and it ends the argument about which indicator is best by showing there was nothing to be best at.

Sources. Peter C. Wason, “On the failure to eliminate hypotheses in a conceptual task” (Quarterly Journal of Experimental Psychology, 1960), for the original demonstration that people test a rule by looking for cases that fit it, which is the behaviour the twelve configurations above make cheap. Raymond S. Nickerson, “Confirmation Bias: A Ubiquitous Phenomenon in Many Guises” (Review of General Psychology, 1998), for the survey of where it shows up and for the distinction this lesson leans on between seeking evidence and interpreting it. Halbert White, “A Reality Check for Data Snooping” (Econometrica, 2000), for what happens to a hit rate when it is the best of twelve rather than the only one, which is the correction the worked table above would need before any of its columns could be believed. Campbell R. Harvey, Yan Liu and Heqing Zhu, “… and the Cross-Section of Expected Returns” (The Review of Financial Studies, 2016), for the size of that correction once the number of things tested is counted honestly.

Three indicators at four lookbacks is twelve configurations and about two opinions, and what they disagree about is the lookback rather than the indicator: same lookback, different family, they agree 91 per cent of the time. On 20 of the 39 bars where all twelve are defined, one of them reads bullish and another reads bearish, so choosing is unavoidable and no reading on the chart marks the choice as a choice. None of the twelve beats the tape on its own bars. That is the last thing this module has to say about indicators, and it is the thing that makes the rest of it usable: an indicator is a filter with a delay you can compute, a level you can solve for, and a base rate you can measure against, and every one of those is a number rather than an opinion. Lesson 53 leaves the single chart entirely and asks who was on the other side of every fill in it.

Educational only. Trading involves substantial risk of loss. Not financial advice. Past performance does not guarantee future results.