Detecting a Regime Change

Finding where a finished series changed and deciding that it has just changed are two different problems, and the one you actually have is the second. Everything about it is a trade between lag and false alarms. On the sixty closes below, one bar of confirmation halves the number of declared changes from eight to four, and five bars halves it again to two — and how much of the move you end up inside does not fall as you add confirmation. It goes seven, four, three, two, and then ten out of ten.

Prerequisites: Lesson 36, whose ratio, series and thresholds this lesson runs at the right-hand edge instead of over the whole sample, and lesson 32, which already priced the same instrument on swings: a turn confirmed after two bars is a turn you learn about two bars late.

The same series, read one bar at a time

Take lesson 36’s sixty closes and its ratio at a threshold of 0.4, and walk forward. At each bar the raw signal says trending if the reading is above 0.4 and not otherwise; a change is declared only once the new answer has held for the confirmation length. Below, the ten-close window, which is the responsive one.

The last column needs a definition and an honest label. Lesson 36’s twenty-close reading put the rally at bars 44 to 53, and that is the run being scored here. It is not ground truth — it is one setting’s answer, adopted as a yardstick so that the columns can be compared to something fixed.

Confirmation, 10-close window	Changes declared	Trend first declared	Of the ten trend bars, inside for
1 bar	8	bar 34	7
2 bars	4	bar 35	4
3 bars	4	bar 36	3
4 bars	4	bar 37	2
5 bars	2	bar 38	10

The second column is the trade working exactly as advertised. One bar of confirmation cuts the declared changes from eight to four; five bars cuts them to two. The third column is what that costs: each extra bar of confirmation pushes the declaration one bar later, which is the only thing it can do.

The fourth column is the one nobody warns you about. Going from one bar of confirmation to two does not improve your position in the move — it wrecks it, from seven of the ten trend bars down to four. Three bars gives three, four bars gives two. And then five bars gives all ten.

The reason is in the shape of the signal. Between bars 34 and 53 the ten-close reading is above the threshold for six bars, below it for four, above for one, below for two, above for one, below for one, and above for the last five. The longest stretch below the threshold in that whole span is the four bars at 40 to 43. Any confirmation of four or fewer is knocked out there and then has to fight its way back in through the one-and-two-bar chop that follows, which is why it spends the strongest part of the rally outside. Five bars is longer than every dip in the span, so it never leaves at all.

Nothing about that is intuitive, and nothing about it generalises. It is a property of where the dips fell on this series. The point is not the shape of the column; the point is that the column is not monotonic, so you cannot reason your way to a confirmation length. You have to run it.

The dial that is not even written down

There is one more setting in that table, and it is not the window, the threshold or the confirmation. Bar 43 reads exactly 0.4. The rule above says trending when the reading is above the threshold, strictly, so bar 43 counts as not trending — but a rule that said “at or above” is just as ordinary, and nobody who quotes a threshold ever says which they mean.

There is a second edge underneath that one, and it is worth seeing because it is the same kind of thing. Bar 43’s ratio is exactly two fifths, and in ordinary floating-point arithmetic it does not come out that way: summing the ten steps in the order they occurred gives 0.3999999999999988, which sits below the threshold under either rule. So the boundary this paragraph turns on is invisible to a spreadsheet, and two people running the same rule on the same closes can disagree because of how their software accumulated a sum. That is not a reason to distrust the arithmetic. It is a reason never to build a decision on a reading that is sitting on its own threshold.

One bar, one boundary, and two rows of the table move. Under “at or above”, two bars of confirmation declares six changes rather than four and holds six of the ten trend bars rather than four; four bars of confirmation declares two changes rather than four and holds all ten rather than two. Same series, same window, same threshold, same confirmation lengths. The only thing that changed is whether one reading of exactly 0.4 was counted in.

The same table with a longer window

Confirmation, 20-close window	Changes declared	Trend first declared	Of the ten trend bars, inside for
1 bar	2	bar 44	10
2 bars	2	bar 45	9
3 bars	2	bar 46	8
4 bars	2	bar 47	7
5 bars	2	bar 48	6

Two changes at every confirmation length, and capture that falls one bar at a time. The long window has nothing left to confirm, because averaging over twenty closes has already done the confirming: a dip that knocks the ten-close reading below the threshold does not move the twenty-close reading enough to matter. A longer lookback and a confirmation delay are the same instrument bought two different ways, and buying both is paying twice.

Set the two tables side by side and one row stands out. The ten-close window with five bars of confirmation declares two changes, the same as any row of the second table, and declares the trend at bar 38 — six bars earlier than the quietest twenty-close setting manages. On this series, the responsive measurement with a long confirmation beat the slow measurement outright. That is a real result and it is one series, sixty closes, one instrument. Take it as a reason to run the comparison on your own data, not as a recommendation.

What a false change actually costs

The money cost of a wrongly declared change is a real number and it belongs to whatever you did about it, which this lesson cannot know. The cost that can be stated is the one lesson 36 was counting in.

The ten-close window with one bar of confirmation declares eight changes across the fifty bars it can be read on: a change of strategy roughly every six bars. Whatever your two strategies are, neither ever accumulates a run of trades under a single set of conditions, because the label keeps moving. Lesson 19’s arithmetic then bites twice over: the trades are already scarce, and now they are also split across two piles that keep swapping. A record like that cannot answer a question about either strategy, and the reason is not the market. It is a confirmation length of one.

Which turns the choice into something you can actually make. A setting that declares a change every six bars is not a sensitive setting; it is a setting that has destroyed its own evidence. A setting that declares two changes in fifty bars gives each label enough consecutive bars to hold a sample, and the lag it charges for that is four bars, which is a number you can look at and price.

Problems

1. Run your own detector forwards and count the changes. On sixty closes of your own instrument, compute lesson 36’s ratio at a window and threshold you pick, then walk forward bar by bar and write down every bar at which the label flips, with no confirmation at all. The count is your false-alarm budget at the most sensitive setting there is. Divide sixty by it: that is how many bars each of your labels survives on average, and if the answer is under about ten, no strategy of yours will ever collect a sample under either label.
2. Buy the quiet and price it. Repeat with a confirmation of two bars, then five. Record three things each time: how many changes were declared, how many bars later the first change was declared, and how many bars of the biggest move you were on the right side of. Expect the third column to misbehave. If it falls and then rises, you have found the same non-monotonicity as above, and the bar count of your longest mid-move dip is the reason.
3. Score yourself against the shaded chart, then against the tape. Take a chart where you can see the regimes clearly and mark, from the finished picture, where each one began. Then hide everything to the right of each mark and ask what your rule would have said standing at that bar. The gap between the two answers, in bars, is your detection lag, and it is the number that belongs in any decision about position size at a turn. How to Collect a Base Rate is how the count is kept honest.

Sources. E. S. Page, “Continuous Inspection Schemes” (Biometrika, 1954), for the sequential problem stated properly and for the cumulative-sum scheme that answers it — a procedure whose entire design is the choice between how fast it notices and how often it is wrong. G. Lorden, “Procedures for Reacting to a Change in Distribution” (The Annals of Mathematical Statistics, 1971), for the result that the trade is a frontier rather than a matter of skill: bound the false-alarm rate and you have bounded the detection delay from below. Jushan Bai and Pierre Perron, “Estimating and Testing Linear Models with Multiple Structural Changes” (Econometrica, 1998), for the retrospective problem done properly, which is worth reading precisely because it uses the whole sample and says so — the contrast with the live problem is the lesson.

Two lessons have now been spent measuring a series of sixty bars, and neither has said what a bar is. Every number in both of them — the ratio, the window, the confirmation, the lag — is quoted in bars, and a bar is an interval somebody chose. Lesson 38 is about that choice: what a timeframe actually is, what changes when you change it, and which of the quantities you have been reading survive the switch unaltered and which are artefacts of the interval.

Educational only. Trading involves substantial risk of loss. Not financial advice. Past performance does not guarantee future results.