A subscriber recently asked a very reasonable question: If patterns are everywhere, how do we know which ones matter?
Good question. A better question, perhaps, than asking which direction the S&P will move tomorrow. And certainly easier to answer than why anyone would pay seven dollars for a bottle of water at an airport.
Imagine tossing a coin ten times and getting eight heads. That seems unusual. Now imagine tossing a coin ten times a million times. Somewhere in those million experiments you will find ten heads in a row. Several times, probably. Did the coin suddenly discover a preference? Was there a mysterious force pulling the metal toward heads? Or did you simply perform enough experiments to find something that looked extraordinary?
This is the problem with market research. Search long enough and you will find something. Search through enough markets, enough dates, enough indicators and enough combinations of parameters, and you will find something that looks spectacular.
Finding it is not the difficult part. Deciding whether it means anything is.
We tend to think a pattern is valuable because it repeats. But repetition alone is not evidence of a usable forecasting relationship. The days of the week repeat. The phases of the moon repeat. My dog asks for dinner at roughly the same time every day. Only one of those has a direct and easily demonstrated relationship to the food bowl.
A useful market hypothesis needs a mechanism or at least a relationship that survives attempts to break it. Divide the historical observations into different periods. Change the beginning and ending dates. Check whether a small number of unusual trades account for most of the apparent success. And, perhaps most importantly, test the idea on observations that played no part in developing it.
This last point is where otherwise intelligent people get into trouble. We build the model with one eye on the answers and then congratulate ourselves for finding them. It is the financial equivalent of giving a student tomorrow's exam today and announcing that we have discovered a genius.
A pattern may exist in historical data without being stable enough to exploit. A stable relationship may exist without producing a profit after costs. And a profitable historical relationship may disappear once enough people begin using it.
Three separate questions. Three separate answers. Yet they are routinely treated as one.
Our objective should not be to find the prettiest result. It should be to identify relationships that remain interesting after we have made every reasonable effort to prove them wrong.
That may sound unnecessarily pessimistic. I prefer to think of it as scientific housekeeping. If the pattern is real, a little cleaning shouldn't make it disappear.
Services
