Positioning Data

The positioning report is real, it is free, and it is worth reading. Almost everything said about it counts one fact twice. Long open interest equals short open interest in every futures market, always, so the net positions of the reporting categories sum to exactly zero — which makes commercials at a three-year short extreme and speculators at a three-year long extreme one sentence written down twice. Even if the smallest category swung as violently as the largest, the two headline series would still correlate at -0.71 by accounting alone; in practice they run nearer -0.95, which by lesson 45’s arithmetic makes two confirming indicators 1.03 independent readings. The index everyone quotes is worse than that. It is a range statistic pinned by two observations out of a hundred and fifty-six, it calls a reading extreme on 28 per cent of weeks where a true percentile calls it on exactly 20, and one new record week moves a historical reading by as much as 44 points where the percentile moves it by 1.6. What survives is smaller than the folklore and better than nothing: a mandatory, audited census of who holds what, three sessions stale at publication and four by the time you can trade on it.

Prerequisites: Lesson 45, whose effective count is what settles the question of how many indicators are on this page, lesson 44, which supplies the arithmetic that prices the reporting lag, and lesson 19, which prices what it would cost to find out whether any of it works.

What the index is made of

The formula in general use takes the current net position, subtracts the lowest reading of the past three years, divides by the difference between the highest and the lowest, and multiplies by a hundred. Two observations out of the hundred and fifty-six in a three-year window set the entire scale. The other hundred and fifty-four only decide where a reading falls between them.

Turn the module’s sixty closes into a positioning series by a published rule so the arithmetic can be checked: net contracts equal the close minus 100, multiplied by 10,000. That gives a low of -18,000 and a high of 71,000. The percentile column is the share of the sixty readings sitting strictly below the one being scored, which is why the highest of them reads 98.3 rather than 100. Below, seven of those readings scored two ways, and then scored again after one new week arrives at 142,000 contracts, double the old high, which is what a genuine three-year record looks like when it turns up.

Net position	Index	True percentile	Index after the new record	Percentile after
-18,000	0.0	0.0	0.0	0.0
-12,000	6.7	1.7	3.8	1.6
0	20.2	16.7	11.2	16.4
30,000	53.9	50.0	30.0	49.2
60,000	87.6	80.0	48.8	78.7
69,000	97.8	95.0	54.4	93.4
71,000	100.0	98.3	55.6	96.7

Three things fall out of the table. Start with the disagreement between the two ways of scoring the same reading: across the whole series the index and the percentile differ by a mean of 5.15 points and by as much as 11.5. The direction is not random: the index reads higher than the reading deserves on 59 of the sixty weeks. Through the middle of the range that overstates how extreme a reading is. At the bottom the same bias runs the other way in effect — a net position at the 1.7th percentile, rarer than all but one week in sixty, is reported by the index as 6.7, which reads as unusual rather than as almost unprecedented.

Second, the index calls extremes more often than extremes occur. A true percentile is at 80 or above on exactly a fifth of readings, because that is what a percentile is for. The index on this series is at 80 or above on 28.3 per cent of them. The band people are told to act on is 42 per cent wider than the word suggests, and every reading inside the extra width arrived there by the shape of the range rather than by being rare.

Third, and this is the part that should settle the argument. One new week at 142,000 changes nothing about any earlier week: the same positions, held by the same firms, against the same prices. Recompute anyway, because the formula demands it. The number of the original sixty weeks reading 80 or above falls from 17 to none. The week that read 87.6, comfortably inside the band, now reads 48.8, which is the middle of the range. Its true percentile went from 80.0 to 78.7. One observation moved the index by 39 points and the percentile by 1.3, and the largest index shift anywhere in the series was 44 points against the percentile’s 1.6 — a factor of nearly 28 in how much a single new observation is allowed to rewrite the past.

What survives

Quite a lot, once the double-counting and the index are gone. The report is a census rather than an estimate, which no other positioning data available to a retail trader can claim. Its level and its week-to-week change are real information about who is holding what, even after the lag. And its most defensible use is the one the folklore states almost by accident: it is a veto rather than a signal. It never tells you to buy anything. It tells you when the thing you were about to do puts you on the opposite side of the largest reported holders in that market, at a moment when they are as committed as they have been in years.

A veto is a filter, and lesson 39 already priced what a filter has to clear: it improves your expectancy only if the trades it deletes contained a higher share of your losers than of your winners, and it improves your money only if that ratio beats your profit factor. Whether this particular filter clears those two bars on your record is a question your record can answer and nobody else’s can. Until it has, read the percentile rather than the index, divide the position by open interest before calling anything crowded, use the disaggregated report rather than the legacy three lines, and treat the whole thing as one reading rather than two.

What this does not settle

That the categories are kinds of trader. They are self-declared reporting buckets, and the commercial bucket in the legacy report mixes producers with swap dealers who are hedging exposure taken from investors. That is not a subtle objection: it is the reason the Commission itself began publishing a disaggregated version in 2009. Reading the three-line report and describing the first line as the physical market is describing something the regulator has already told you is two different things added together.

That the futures position is the whole position. The report covers futures, and in some markets futures plus options on futures. It says nothing about swaps, physical inventory, forward sales, or anything in the cash market. A miner short a large futures position may be long far more of the metal in the ground, and the net of the two may be flat or the other way round. What you are reading is one leg of a book you cannot see.

That the identity makes the report worthless. It makes two readings into one reading, which is a large correction and not a demolition. One honest reading of who holds what, published under legal compulsion, is more than most markets give you. The mistake is arithmetic rather than epistemic: the error is in counting the confirmation, not in consulting the report.

That the double-counting argument survives the report this lesson tells you to read. It binds hardest on the legacy three lines, where two categories are large and the third is a remainder. The disaggregated report has five, and five series summing to zero carry four independent numbers rather than two. If those five were of comparable size and otherwise unrelated, the accounting floor on any pair would be minus one divided by four, which is -0.25, and two readings at that correlation are 1.6 independent readings rather than 1.03. The identity never goes away and it never stops deleting something. But on the report this lesson recommends it deletes a good deal less than the headline above, the categories are nowhere near equal in size so the truth sits somewhere between the two figures, and where it sits is measurable on the published series and is not measured here.

That these sixty numbers are anybody’s positioning series. They are the module’s closes rescaled by a rule printed above, so that every figure in the table can be reproduced. Sixty readings is also not the hundred and fifty-six a three-year window holds, and a longer window would soften the index-versus-percentile gap without changing its direction or its fragility. The arithmetic transfers; the numbers do not.

That any of this settles whether extremes precede reversals. It does not, and neither does anything else in print. The hit rates quoted for extreme positioning trace back to nobody, and the single episodes everyone cites are single episodes. Lesson 19 prices what it would cost to establish the answer on your own record, and the answer is more weekly observations than most people will collect in a decade of trading one market.

Problems

1. Check the identity yourself, once. Pull one week of one market from the Commission’s own site, add the net positions of every category including the non-reportable remainder, and confirm the total is zero. It will be. Then write down what that means for the two numbers you were about to treat as independent: you have two degrees of freedom across three lines, and any confirmation you find between the first two was there before the market opened.
2. Compute both statistics on the same reading. Take three years of weekly net positions for one market you follow. For the most recent week compute the index in general use and the true percentile rank of that reading among all of them, and write both numbers down side by side. Then divide the position by that week’s open interest and write that down too. You now have three numbers describing one position, of which the public conversation quotes the least robust.
3. Count what the veto would have removed. Go through your own record and mark every trade that put you opposite the largest reported category while it sat above the eightieth percentile. Count the trades and total the money in that set, and in everything else. That tells you what the filter would have done to you rather than what it did to somebody in an article, and lesson 39 gives you the two conditions the result has to clear before the filter is worth running.

Sources. Holbrook Working, “Futures Trading and Hedging” (The American Economic Review, 1953), for the demonstration that hedging is discretionary and carries a price view, which is the assumption the whole case for following the commercial category rests on. Commodity Futures Trading Commission, the Commitments of Traders explanatory notes and the disaggregated report introduced in 2009, for what the categories are, what the reporting thresholds do, and the regulator’s own reason for splitting the commercial bucket. Changyun Wang, “The Behavior and Performance of Major Types of Futures Traders” (The Journal of Futures Markets, 2003), for one of the few serious attempts to measure whether the categories forecast anything. Michael S. Haigh, Jana Hranaiova and James A. Overdahl, “Price Dynamics, Price Discovery and Large Futures Trader Interactions in the Energy Complex” (Commodity Futures Trading Commission, 2005), for the finding that the speculative category followed prices rather than leading them, which is the opposite of what the popular reading assumes.

Positioning is a census of who holds what. It says nothing about why all of them are holding it at once, and lesson 45 named the quantity that decides how much of a book is really one position: a common factor whose variance rises and falls. That factor has a name and a publication schedule. Lesson 47 takes up the macro cycle: which released numbers are new information and which are arithmetic on figures already public, why a release moves price by its surprise rather than by its level, how to compute that surprise from what was already quoted, and why the same figure moves the same market in opposite directions in different years.

Educational only. Trading involves substantial risk of loss. Not financial advice. Past performance does not guarantee future results.