How to create DataSeries "Correlation"
Author: algoapi
Creation Date: 3/23/2009 10:33 AM
profile picture

algoapi

#1
I tried to generate the correlation dataseries using the following function:

Correlation(double[] x, double[] y, int n)

But the sytem returns the message that this is not recognized. Can anyone show me an example on how to generate Dataseries Correlation (for Pearson correlation)?

Many thanks!



profile picture

Eugene

#2
Here's an example:
Correlation

Requires Community.Components (Extensions > Get Extensions).
profile picture

algoapi

#3
Thanks Eugene.

I did tried to add the following at the beginning:
"using Community.Components;"
but still didn't work.

All I wanted to do is to generate correlation dataseries between two stock time series.

One thing makes me a bit confused is that the other math functions such as LinearRegSlope works well without additional requirements (extensions), while the correlation function requires the extension, which I still couldn't figure out how to get it work. Isn't it more convenient to have all the commonly used math functions built in the WL platform?
profile picture

Eugene

#4
QUOTE:
I did tried to add the following at the beginning:

Did you follow the link to install the Extension from this site? You need to restart WL upon installing like instructed here. Upon restarting, the Extension will become available to Strategies.

QUOTE:
but still didn't work.

If something doesn't work, how could we help without at least having a precise error message text?

QUOTE:
One thing makes me a bit confused is that the other math functions such as LinearRegSlope works well

That's because they are included in the WealthScript/WealthLab.Indicators namespace. And if some function isn't available natively, we can leverage Wealth-Lab with the power of .NET. Extensibility is the key.

QUOTE:
Isn't it more convenient to have all the commonly used math functions built in the WL platform?

For the reason above it's neither possible nor practical to include everything in the package. Look, there are no less than 4 .NET compliant math libraries:

Open Source .Net math libraries

As to correlation, there will always be someone who would prefer some other implementation: DCorrelation, SCorrelation
profile picture

algoapi

#5
Thanks again for the detailed answer :-)
I didn't mean to pick on the platform, actually I think WL is a very good platform.
I didn't post exact message that's because my current PC cannot run WL lively due to firewall. I posted the question based on my experience a few days ago.

For the last point, I still think the Pearson Correlation is the most commonly used math function in many areas. When people mention "correlation" typically that should refer to Pearson Correlation. Actually most other platforms all have this function as one of the basic math funtion built-in.

Anyway, the above is just my advice. I appreciate your hard work, this is a great platform overall.
profile picture

DrKoch

#6
algoapi,
please note: it is rather meaningless to send prices (for example closing prices) into the correlation function. Will result in GIGO.

You'll have to calculate first differences (TA people call this "Momentum" a formidable misnomer) make sure to use percentages or even better logarithmic returns - just be warned...
profile picture

algoapi

#7
Thanks DrKoch for the note, I totally agree.
In my code I actually tried to get the correlation between two stock's percentage return series, not on the prices directly.
profile picture

Eugene

#8
Thanks for the good words. I understand your point.

(Let's ask Fidelity, do they plan to make Correlation available again natively like it was in WL3/4 in a later release?)
profile picture

algoapi

#9
Hi Eugene,

I followed the instruction and installed the community.components extension successfully (copy of screenshot as below)

Community Components Library
Install File: Community.Components.dll
Succeeded

---------------------------

Then I tested the following sample code to calculation correlation:

CODE:
Please log in to see this code.

When I run the code, the system gives me error message as below:

error CS0103 @ (17,22): The name 'Correlation' does not exist in the current context.

Could you advice what I missed here? Thanks
profile picture

Eugene

#10
Have you seen the code example mentioned in my reply @ 3/23/2009 11:08 AM? In short, you haven't created the data series as illustrated in the example.


Please use the CODE tags when posting code.
profile picture

ss161

#11
i needed the function also. the community component function takes an array of doubles and returns the correlation. to calculate the correlation that way for each bar in a price series seems inefficient (each time it is called, it "forgets" all the prior overlapping data items and rebuilds them. i wrote my own utility based on the same logic supplied in the community component and it seems to work -- although i expressly state that this comes with no warranty or assurance that the results are correct. please test it if you are going to use it.

here is the utility class code, followed by a simple program to show how it is used ( i too generally use log(price) series or log(dif(price) series, but am testing it via a simple price series ).

CODE:
Please log in to see this code.


and the program to test it

CODE:
Please log in to see this code.
profile picture

algoapi

#12
Thanks a lot ss161. I will try your program later. Cheers.
profile picture

Eugene

#13
Steve, thanks for sharing. A good idea would be to enhance the Wiki description with your work and include this indicator in Community.Indicators.
profile picture

ss161

#14
you're certainly welcome to do so -- assuming that would be something that you or Cone would do?
profile picture

Eugene

#15
Yes, we'll take care of it.
profile picture

Eugene

#16
Steve, it looks like your data series gives a different value compared to the WL4's Correlation function. Am I missing something?
profile picture

ss161

#17
i'll compare the two later today and come back. thanks for pointing it out.
profile picture

DartboardTrader

#18
The below matches Excel's output, and these are useful for anyone converting from Excel-based lifeforms.

Clean and optimize, if needed.


CODE:
Please log in to see this code.


Usage:
CODE:
Please log in to see this code.


Regards,
--Mike
profile picture

Eugene

#19
Correlation galore! Thanks for sharing, Mike.
profile picture

ss161

#20
after painfully staring at the code for a while, i realize that
CODE:
Please log in to see this code.

does not work; there needs to be a double loop summing all (x - xmean) and (y - ymean)

the following seems to work, i've tested against using the Community.Component to calculate correlation and plot the two on top of each other on a chart.

here is the amended code -- although i'm not sure there is any efficiency gain over Mike's or just using a wrapper around the Community.Component.

CODE:
Please log in to see this code.


Here is a program that tests it, using the input dialogue box that you helped me with yesterday:

CODE:
Please log in to see this code.
profile picture

Eugene

#21
Perfect. Thanks.
profile picture

Eugene

#22
Both functions added to Community.Indicators 2009.04. Thanks gentlemen.
profile picture

ajrdwv07

#23
quick question. how can I create autocorr? that is the correlation(close, close t-5, 40) for example, that is the corr between the eod and the eod 5 days ago. thanks.
profile picture

Eugene

#24
Please strike F11 in Wealth-Lab to open the QuickRef tool and navigate to "DataSeries Object" to read the description. Here's an example of a shifted Close applied to Correlation DataSeries:

CODE:
Please log in to see this code.
profile picture

ajrdwv07

#25
useful. many thanks.