Markets Have Modes
The crudest regime measurement there is takes one ratio: how much of the distance price travelled ended up as net progress. On the sixty closes below it separates the sideways stretch from the rally completely, with no overlap. Then change the window from ten closes to thirty, and the two readings of the same series disagree about which regime it is in on twenty-two of thirty bars.
Prerequisites: Lesson 19, which is where the arithmetic this lesson spends comes from — how many trades a question about your own record actually costs — and lesson 34, whose sixty closes are the series measured here.
Every lesson in the last module ended in the same place: the reading depends on a setting nobody displays. This module asks a different question. Not what a pattern means, but when it means anything — because a rule that pays in one market condition loses in another with nothing about the rule having changed, and that is the most widely agreed proposition in this whole subject.
It is also the vaguest. A regime is a claim that the last stretch of bars was produced by a different process from the stretch before it, and processes are not things you can see. What you can see is prices. So every regime tool in existence is a rule for turning prices into a label, and the label is only as good as the rule.
The crudest version
Start with the least elaborate measurement anyone has proposed, because it is the one whose parts you can count. Take a window of closes; a twenty-close window here means the twenty steps ending at the bar you are reading, which is twenty-one closes in all. Add up the absolute size of every step from one close to the next, which is the distance price travelled. Divide the net move from the first close to the last by that total. The answer is between nought and one: one means every step went the same way, and nought means price ended exactly where it started having gone a long way to get there.
That is the whole measurement. Perry Kaufman published it as the efficiency ratio and used it to make a moving average adapt; the arithmetic is older than the name and simple enough that you can do a window by hand in a minute. One quantity, one setting.
Now the reason it is not as crude as it looks. Every regime indicator in common use is asking the same question with more parts bolted on. ADX compares directional movement to total movement and then smooths the comparison twice, which adds two more settings. Bollinger Band width divides a volatility band by its own centre line, adding a period and a standard-deviation multiplier. An ATR ratio compares a short average range to a long one and needs both lengths chosen. Each is defensible; none of them removes the problem the crude version makes visible, because each one adds settings rather than taking any away.
The mistake in adding them together
There is a common construction worth naming before the numbers, because it is wrong in a way that is easy to miss: the composite regime score, where you award a point for ADX above 25, a point for ATR above one and a half times its average, a point for band width above some percentage, and total them.
Trend and volatility are not the same axis. A chaotic, news-driven session can produce a high ADX, an elevated ATR and a wide band all at once, which is three points and a reading of “strongly trending” on precisely the kind of day the same page will tell you to stand aside. Scoring the two separately fixes it: a trend reading and a volatility reading, side by side, and any rule that wants to size up has to satisfy both. Any version of the score that needs a volatility override bolted on afterwards is telling you the single number never worked.
Sixty closes, one ratio, three windows
The series is the one this module has been building. Lesson 32 gave the highs and lows of the first twenty bars, lesson 33 added their opens and closes, lesson 34 carried the closes out to sixty and lesson 35 supplied the forty highs and lows that were still missing, so the sixty bars are now complete and every number in the last five lessons has come off them. This one needs the closes alone: nothing here calls for highs, lows or an indicator library. Read it at each bar over the previous twenty closes and the picture is unambiguous. Through bars 21 to 42 — the stretch that falls to 98.2 and grinds back — the reading never rises above 0.22. Through bars 44 to 53, the rally, it never falls below 0.42, peaking at 0.60 on bar 47. Bar 43 reads 0.32 and is the only value between the two groups.
That is a clean separation with a gap in the middle, produced by dividing one number by another. The crude measurement found the two stretches that are actually there, and it did so without a single parameter beyond the window length. Anyone who tells you regime detection requires a suite of indicators has not tried the arithmetic.
Now change the window. Below is the count of how many of the thirty bars that all three windows can be read on — bars 31 to 60 — come out as trending, at four thresholds anyone might pick.
| Window | Trending at 0.2 | Trending at 0.3 | Trending at 0.4 | Trending at 0.5 |
|---|---|---|---|---|
| 10 closes | 21 of 30 | 20 of 30 | 13 of 30 | 8 of 30 |
| 20 closes | 19 of 30 | 13 of 30 | 10 of 30 | 4 of 30 |
| 30 closes | 13 of 30 | 10 of 30 | 1 of 30 | 0 of 30 |
Read the corners first. The same sixty closes are trending on twenty-one of thirty bars under one ordinary setting and on none at all under another. Both settings are in use, and neither is eccentric: a ten-bar lookback with a threshold of a fifth is as defensible as a thirty-bar lookback with a threshold of a half.
The middle of the table is worse than the corners, because the corners at least disagree in an orderly way. Take a single threshold, 0.3, and compare the ten-close reading against the thirty-close reading bar by bar. They agree on eight of the thirty. Two windows, one series, one threshold, and they give opposite answers more often than they give the same one — not because either is broken, but because a ten-close window inside a long sideways drift keeps finding short runs that the thirty-close window averages away.
What the classifier has to beat
Suppose you now grade your regime rule against what the market did next, and it is right seven times out of ten. Before that means anything, work out what a rule that ignores the market entirely would have scored.
On this series, at a twenty-close window and a 0.4 threshold, ten of the forty readable bars are trending. So the answer “ranging”, given every single day without looking at anything, scores thirty out of forty: seventy-five per cent. A classifier at seven out of ten is not beating that; it is losing to it. The number that matters was never the accuracy, it is the accuracy above the base rate, and the base rate is a thing you have to compute before you can know whether you have learned anything at all. How to Collect a Base Rate is the procedure, and the first problem below applies it here.
What a filter costs, in the only currency you have
Every regime rule ends the same way: in these conditions, do not trade. That is the point of it, and it is not free, though the price is rarely stated because it is not paid in money.
Lesson 19 established the exchange rate. To establish that a strategy with a 0.35R edge has any edge at all takes about 143 trades, which at four trades a week is a little over eight months. A filter that cuts your trade rate does not change that number of trades; it changes how long they take to arrive.
| Share of trades the filter keeps | Trades a week | Weeks to 143 trades | Roughly |
|---|---|---|---|
| All of them | 4.0 | 36 | 8 months |
| 70% | 2.8 | 51 | 12 months |
| 50% | 2.0 | 72 | 16 months |
| 30% | 1.2 | 119 | 2 years 3 months |
| 20% | 0.8 | 179 | 3 years 5 months |
| 10% | 0.4 | 358 | 7 years |
A filter that keeps three trades in ten — which is an ordinary strength for a regime gate, not an extreme one — turns an eight-month question into a twenty-seven-month question. Both halves of that are true at once. The filter may well be saving you from the conditions your rule was never built for, and it is simultaneously making it three times slower to find out whether your rule works at all. Only the first half ever appears in the pitch.
The one thing you can measure this month
Which leaves a real problem: for the whole first year of running a regime filter, your results cannot tell you whether the filter is any good. There is one measurement that can, and it works because it is not about outcomes.
Count the trades you took whose conditions you had written down in advance, and divide by all the trades you took. That fraction is compliance, not performance. It needs no sample, because there is no estimate in it and nothing to converge: after five trades it is already exact. It answers a question you fully control — did you follow your own rule — and it is knowable tonight, where the question of whether the rule pays is knowable in two years. For the first stretch of running any filter, that is the only honest feedback the method can produce.
What this does not settle
Which window and which threshold are right. The table has no winning row, and it was built to make that visible rather than to hide it. Every setting in it is in ordinary use. What the table settles is that the word trending does not name a state of the market until a window and a threshold have been chosen, and that most people who use the word have chosen neither.
Whether the label predicts anything. Everything above measures the series and nothing above measures what happened next. That a twenty-close ratio separated the sideways stretch from the rally is a statement about a description, not about a forecast — and note when it separated them, which was afterwards. At bar 44 the reading is 0.48 and you have no way of knowing whether the rally has ten bars left or one.
That four regimes is the right number. Trending, ranging, volatile and compressed is a common carve-up and a useful one, but it is a choice of vocabulary, not a finding. The formal literature models regimes as a hidden state with a fixed number of values, and the number is an assumption of the model. Two states, three states and four states all fit somebody’s data.
That the crude ratio is as good as the elaborate tools. Nothing here tested that. What was shown is that it is legible: you can see every setting it has, which is one, and watch what happens when you move it. ADX and band width may well read the market better. They will also hide four settings where this hides one, and none of the arguments above gets easier when the count goes up.
That volatility and trend can be scored together. They cannot, and the composite score above is the standard way of getting this wrong. Keep them as two readings.
A regime label is a summary of the recent past with a threshold attached. That can be worth having. It becomes dangerous at the moment you forget that you chose the threshold, and it becomes a different problem altogether at the moment you stop asking what the last twenty bars were and start asking whether the last two have changed.
Problems
- Compute your base rate before you grade anything. Take sixty consecutive closes from your own instrument and run the ratio at a twenty-close window. Count how many readings clear 0.4 and how many do not. Whichever answer is more common is what a rule that never looks at the market would score, and that is the number your regime rule has to beat. Most people have never seen it, and a good many regime rules do not clear it. How to Collect a Base Rate is how the count is kept honest.
- Run two windows and count the disagreements. On the same closes, compute the ratio at ten and at thirty, label each bar trending or not at a single threshold you pick in advance, and count the bars where the two labels differ. That count is not a flaw in your data. It is the size of the choice you are making when you pick a lookback, and it is invisible until you compute it.
- Price your own filter before you adopt it. Take your last three months of trades and mark each one as inside or outside the conditions your strategy actually needs. The fraction inside is what the filter keeps. Divide your usual trades per week by it, then divide 143 by the result: that is how many weeks a filtered record needs before it can tell you whether you have an edge. Write that number down next to the reason you wanted the filter, and decide with both in front of you.
Sources. James D. Hamilton, “A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle” (Econometrica, 1989), for what a regime is when it is stated properly: a hidden state that is never observed, only assigned a probability, with the number of states fixed by the modeller rather than discovered in the data. Perry J. Kaufman, Smarter Trading (McGraw-Hill, 1995), for the efficiency ratio as published, including its intended use — adapting a moving average’s speed rather than issuing a label. Andrew Ang and Allan Timmermann, “Regime Changes and Financial Markets” (Annual Review of Financial Economics, 2012), for the review of what the estimated models actually find, including the part this lesson keeps returning to: identifying a regime after the fact is a far easier problem than identifying it while you are in one.
The next lesson takes the harder half. Labelling a stretch of past bars is a description, and the ratio above did it cleanly; deciding that the regime has just changed is a decision made at the right-hand edge, where the only bars available are the ones already behind you. Lesson 37 makes that decision on this same series, where one bar of confirmation halves the number of changes you declare, and where the share of the move you end up keeping does not behave the way anyone expects it to.
How Long Until You Know
The arithmetic a regime filter spends, and where 143 comes from.
Read Lesson →Volatility as a Quantity
The other axis, measured on its own terms rather than added in.
Read Lesson →Detecting a Regime Change
Labelling the past is easy; noticing the turn is not.
Read Lesson →Educational only. Trading involves substantial risk of loss. Not financial advice. Past performance does not guarantee future results.
💬 Discussion (0 comments)
Loading comments...
Ready to Trade with Signal Pilot?
Apply your trading education with professional indicators and real-time market analysis tools.
Back to Signal Pilot →