Hello Todd,

One thing I've gotten curious about is how prices (or dollar-volumes) jump between close one trading day to the open of the next. I've attached a very simple-minded algorithm that illustrates the idea. Basically, if the dollar-volume of SPY drops by more than a set percentage between close and open, I buy SPY and then sell it the next day, at opening. In the example, the dollar-volume threshold is set to -70% (for the time period of the backtest, the optimum is actually ~ -30%).

I don't know if the concept has legs, but reportedly there is some historical precedent (see comments by Simon here regarding "buying opening-bell sell imbalances alongside the specialists").

I'm not sure what to do next, but I thought I'd pass the idea along since you seem to be fishing for ideas.

Grant

Hello Todd,
I am not an expert, but I could imagine that FFT could be helpful for some kind of pattern recognition, e.g. looking at volatility patterns.
Best Regards
Fabian

In EE and DSP signals are cyclic in nature. You can model data and extrapolate nicely thanks to this. In some cases you already know what the underlying signal might look like. You can model past market movements too but predicting or extrapolating out of this does not seem to work out as well especially when the movements are random and there are orders of magnitude more factors that can affect it.

The problem with FFT etc. is that they would only show the glaringly obvious cycles (e.g. weekly), other movements (when you look at price versus time, or price versus volume traded) have to not only be regular, but to keep the same PHASE to be visible in a transform like that.

looking at the auto correlation can give you those patterns (if they exist) and be more robust under phase shifts.

@avishalom shalit: Interesting, I can't believe I hadn't thought of using autocorrelation or cross correlation to locate patterns. Have you tried anything of the sort or do you just think it would be an interesting thing to try?

For pattern recognition, you might look into these references:

# References:  
# 1. Ming Li; Xin Chen; Xin Li; Bin Ma; Vitanyi, P.M.B.; , "The similarity metric,"  
# Information Theory, IEEE Transactions on , vol.50, no.12, pp. 3250- 3264, Dec. 2004  
# http://homepages.cwi.nl/~paulv/papers/similarity.pdf  
#  
# 2. Lin, Jessica, et al. "A symbolic representation of time series, with implications for  
# streaming algorithms." Proceedings of the 8th ACM SIGMOD workshop on Research issues  
# in data mining and knowledge discovery. ACM, 2003.  
# www.cs.ucr.edu/~stelo/papers/DMKD03.pdf  

I wrote some code in Quantopian based on zlib for pattern recognition. I'm not sure that I left the code in a useful state...if anyone is interested, I can check it and post an example. Note that although not listed on the Quantopian help page, zlib is available for import. Just use:

import zlib  

Grant

Fourier analysis works best with waves or wavelets that are regular and predictable. The stock market is not either and hence fourier does not provide beneficial results. You might look into spectral analysis and signal extraction - both better options.

The problem you run into is this - there's not much magnitude differentiation across a scale, and both magnitude and phase are transitory between sample sets. That is - can you describe a set of returns via FT? Yes. Absolutely. Perfectly. The problem is how do you then use that to predict the values in some future series?

Since there's no clear link between phase / mag (or real / imag) components between two time periods there's no way to take the very step that would make the setup profitable.

Contrary to those above, I think it's an interesting path for research. But understand that the core problem becomes one of forecasting FT components and it's not at all clear if that is possible. Nonetheless, I don't think its any less an interesting path than looking at raw returns, or the various IIR approaches, or anything else. Maybe there is something you will be able to see in the frequency space that is not visible in the time space.

Just don't expect it to be obvious.

I think a lot of systems that don't do well in time-space might work much better in frequency/volume-space. One of my first quantopian attempts was to try and build equal-volume bars from the minute bars and use those to feed into the alpha model, but if I try it again, I might try and build the bar aggregation infrastructure into zipline (or wait until someone else does). I need the time and permission from my employer before I can put any work into zipline though.

As others have mentioned Fourier analysis is used for periodic signals and has little value as a general predictive indicator. However, you could take it further by using it as an indicator that tells you a market is in a range (as is largely the case for day trading). Set risk/loss targets according to the magnitudes and allocate a small amount of trading capital to scalp with this strategy.

I haven't tried it myself but that's how I would approach it.

I was just thinking about this type of analysis. I am not a quant guy but maybe this idea would help. In my field Fourier analysis is called quantitative EEG. It is a way of making a diagnosis based on relationships that are not apparent to visual analysis. EEG is a measurement of electrical potential between 2 points on the scalp, representing the electrical activity of the underlying brain. So a diagnosis is a predictor that the abnormality discovered will either continue or return in the future. Choosing which variables are relevant is the key to using this idea. So, I would look at the failures first with Fourier analysis and see if the variables for disaster can be illucidated and then ask the question of whether the inverse predicts success. After all, avoiding downturns is as important as predicting winners.
Ron Romanik MD
[email protected]

John Ehlers has done a lot of work in this area. A good place to start http://mesasoftware.com/papers/FourierTransformForTraders.pdf