How to do Fourier Transform to find out the cycle frequencies ?
Author: JohnTong
Creation Date: 1/13/2018 6:34 AM
profile picture

JohnTong

#1
How to do Fourier Transform to find out the cycle frequencies ?
profile picture

Cone

#2
Probably too complex of a discussion for me anymore, but think of it this way... Unless a signal is a pure sine wave, it probably has many frequencies. The output of a Fourier Transform (the frequency spectrum of a signal) is the amplitude of each of the sine waves that compose the signal. The sine wave (a single frequency) with the greatest power (amplitude) would be considered the cycle frequency.

For a demonstration, check out TASC 2008-03 | Measuring Cycle Periods (Ehlers)
profile picture

superticker

#3
I've wondered about the best way to do an FFT (or DFT) in Wealth Lab too. If you're running the Visual Studio IDE, then you could develop a WL Performance Visualizer specifically for this purpose. http://personal.fidelity.com/products/trading/Trading_Platforms_Tools/pdf/Creating-Performance-Visualizers-in-Wealth.pdf

The main reason for implementing this with a Performance Visualizer is because you'll be plotting complex number results (or a power spectrum) on a horizontal frequency scale, and you can only open a Windows.Forms chart window inside a WL Performance Visualizer. You can't do that (easily) within a WL strategy.

Math.Net provides some C# signal processing libraries. https://numerics.mathdotnet.com/
Be sure the read the section on Matrices and Vectors first so you declare your data types right.

From inside your Visual Studio Performance Visualizer project (use the VS Tools menu), you'll need to NuGet (download) "MathNet.Numerics" into your Performance Visualizer project. Now you can add the C# code below to your VS Performance Visualizer project for WL.
CODE:
Please log in to see this code.

Of course, you'll want the windowing functions too (You'll introduce unwanted sampling harmonics if you skip preprocessing the input with a windowing function.), but they are in a different download. From your Visual Studio Performance Visualizer project, you'll need to NuGet (download) "MathNet.Filtering". Now you can add the C# code below to your VS Performance Visualizer project for WL.
CODE:
Please log in to see this code.

There's a nice example coding the FFT with windowing at https://stackoverflow.com/questions/41023808/how-to-properly-run-an-fft-on-a-windowed-set-of-data-from-a-pure-sine-waved

If you get all this working let me know. I haven't had time to explore it yet, but I have thought about it.

---
On a separate note, I see Ehler's solution only explores the dominant cycle (or frequency) and disregards all the components. I do think most of the less significant components are just noise and should be excluded. I'm not sure if only the dominant frequency is important though. But it's an interesting approach.
profile picture

DrKoch

#4
I did it. Several times. The results:

If you understand a price series as a time signal and do a Fourier Analysis, you'll find a nearly perfect pink noise signal.

If you analyze just very short intervals, in some circumstances some "frequencies" may seem to be stronger than others. This is "normal" for a noise signal.

If you try to be clever (I did) and say: Well, if I find some prominent cycle (A strong line in the FFT) I'll "prolong" this curve to predict future prices.
This is quite some effort. The result:
Prediction is not better than random predictions.

In other words: Any "cycle" which is found by an Fourier Transform is just a statistical fluctuation and says nothing about future movements - nothing.
profile picture

superticker

#5
QUOTE:
[Conclusion:] Any "cycle" which is found by an Fourier Transform is just a statistical fluctuation and says nothing about future movements - nothing.
Yes, I was kind of afraid of that. Be sure to try several windowing types because some will create less sampling artifact than others. I kind of like the Gaussian one (shown above) as a good initial choice.

It's really going to depend on the stocks involved. Some stocks are going to have seasonal effects (cycles), but most probably won't. Of the few that are cyclic, they will probably have phase-lag properties. For example, restaurants may trade highest on Fridays than on other days of the week.

I feel kind of bad suggesting this (seems backwards), but maybe this problem should be turned upside down. Identify the predominate noise ranges, then notch filter those bands out (band rejection). Then the residuals are left with only the meaningful signal energies to study.

---
This is off topic, but I think employing some different kinds of nonlinear filters (Median is the most common example) is the best choice here. I don't think linear filters based on complex exponentials (i.e. sinusoids) alone will do justice to this problem. If there's some forum to discuss time series analysis on, I can share more of my thoughts. But this isn't really a Wealth Lab topic.
profile picture

Ben_Zurich

#6
While Dr. Koch without any doubt is one of the most competent contributors here in the forum, I would like to add the following:

The interesting thing about spectrum-extracted cycles in market data is that they tend to cluster around certain number of daycounts, such as e.g. 28-29 days, 120-140 days, 180 days, shift slightly over time (e.g. from 122 days to 127 days etc,), and then disappear for 2-4 years just to reemerge with very similar periods again once a trending phase of the market has terminated.

Interestingly, other "daycount periods" that seem to exhibit more frequent appearances than what would be statistically deemed normal are 1020 to 1080 days, present in many different assets

If someone has an idea how to deal with this, please kindly get back to me. :-)



profile picture

LenMoz

#7
I agree with DrKoch. You can (over)fit cycles to back periods of almost any duration. IMHO, there is no predictive power, however.
profile picture

superticker

#8
QUOTE:
... spectrum-extracted cycles in market data ... tend to cluster around certain number of day counts, such as 28-29 days, 120-140 days, 180 days, shift slightly over time (e.g. from 122 days to 127 days etc,), and then disappear for 2-4 years
A Fourier Transform (FFT/DFT) is going to require about five to ten repeating cycles at a particular frequency to show up at all. From what you describe, I'm not sure you're going to see anything with an FFT. There's just not enough periodicity (signal power) to be detectable with frequency methods. Even an ARMA or ARIMA (Box-Jenkins) time-series model would have some trouble finding it, but an FFT just won't.

Perhaps some type of phase-tracking methodology (e.g. phase-lock loop) might track and flag such an infrequent wave pattern if it fell within its frequency ranges. Phase-lock loops have a front-end low-pass filter.

The Kalman filter for financial apps uses a 3-term 0th-order, 1st-order, and 2nd-order Taylor series to fit a dynamic trend. Have you looked into that? The WL implementation of the Kalman indicator won't do forecasting, which I think is a big omission. But it's still worth a look.

When using the Kalman, if you're working with a really slow trend (cited above) and want to spread the 3-term Taylor series over more days, you could simply switch WL from daily bars to weekly bars to capture the slower trend better. Pretty convenient. Alternatively, for a slightly slower trend, you could smooth the bar data over more daily bars (slow adaptive filter), then feed that into the Kalman.
profile picture

JohnTong

#9
Great thinking. Thanks.

I didn't explore further on this topic, probably due to its complexity (while much easier in matlab), but with limited practical usage.

Thanks.