Calculating correlations - Wealth Lab Forum

Calculating correlations

Author: sedelstein

Creation Date: 3/29/2015 11:26 PM

Wealth Lab » WealthScript (.NET) » Calculating correlations

sedelstein

3/29/2015 11:26 PM

I need help with a problem that I believe likely has something to do with the SetContext method.

I am trying to get correlations between (in this example) 3 symbols over the last 10 years
AAL, AAPL, and ADBE

AAL has 327 bars while AAPL and ADBE both have 2517 bars

In the following code, I am trying to print out ALL the correlations available for the full 10 years
I expected that for the first 2517 - 327 bars I would get the correlations between AAPL and ADBE and the other two would result in NaN's as there is no early data available for AAL

When I fiddle with the SetContext function, I find I need it set to false the first time to get all the history (else I get just 327 correlations for all 3 pairs)

In the second instance, I still only get 327 dates but depending on true or false, I get early or later dates but I am unable to get ALL dates.

Is there a fix you can suggest?

Thanks

CODE:
Please log in to see this code.

Size:

Color:

I need help with a problem that I believe likely has something to do with the SetContext method.

I am trying to get correlations between (in this example) 3 symbols over the last 10 years
AAL, AAPL, and ADBE

AAL has 327 bars while AAPL and ADBE both have 2517 bars

In the following code, I am trying to print out ALL the correlations available for the full 10 years
I expected that for the first 2517 - 327 bars I would get the correlations between AAPL and ADBE and the other two would result in NaN's as there is no early data available for AAL

When I fiddle with the SetContext function, I find I need it set to false the first time to get all the history (else I get just 327 correlations for all 3 pairs)

In the second instance, I still only get 327 dates but depending on true or false, I get early or later dates but I am unable to get ALL dates.

Is there a fix you can suggest?

Thanks

[code]using System;
using System.Collections.Generic;
using System.Text;
using System.Drawing;
using WealthLab;
using WealthLab.Indicators;
using Community.Indicators;		// Correlation, LNRet
using Community.Components;		// Utility, CommentaryWindow

namespace WealthLab.Strategies
{
	public class CorrPair
	{
		public string key;
		public DataSeries Correlation;
	}
	
	public class Ticker
	{
		public string Symbol;
		public int first_actual_bar;
		public int lastbar;
		public DataSeries close;
		public DataSeries ln_returns;
	}

public class MyStrategy : WealthScript
	{
		protected override void Execute()
		{
			string firstRun = ""; string results = "";
			SetGlobal( firstRun, Bars.Symbol );	
			int max_bars = 0;
			
			if( (string)GetGlobal( firstRun ) == DataSetSymbols[0] ) 
			{			
				bool useLNRet = true;		// set to false to use prices, instead of log normal returns, for calculating correlation
				int corr_period =  120;
				string Symbol;
				int symCnt = DataSetSymbols.Count;
				List<Ticker> ticker_list = new List<Ticker>();
				List<CorrPair> corr_list = new List<CorrPair>();

// COLLECT the Data on the symbols
				
				for (int i = 0; i < symCnt; i++)
				{
					try
					{	
						Symbol = DataSetSymbols[i];
						SetContext(Symbol, false);
						Ticker ticker = new Ticker();
						ticker.Symbol = Symbol;
						ticker.first_actual_bar = Bars.FirstActualBar;
						ticker.lastbar = Bars.Count -1;
						ticker.close = Close;
						if (useLNRet)ticker.ln_returns  = LNRet.Series(Bars, Close, 1);
						ticker_list.Add (ticker);
					}
					catch (Exception e)
					{
						PrintDebug(DataSetSymbols[i] + " error: " + e.Message );
						continue;
					}
				}

// Get the Correlations
				for (int i = 0; i < symCnt; i++)
				{
					for(int j = i+1; j < symCnt; j++)
					{
						PrintStatusBar("Symbols: " + (i+1) + "/" + symCnt + " " + ticker_list[i].Symbol + ":" + ticker_list[j].Symbol);
						DataSeries corr = new DataSeries(Bars, "Correlation");

if (useLNRet)	corr = Correlation.Series(ticker_list[i].ln_returns, ticker_list[j].ln_returns, corr_period);
						else			corr = Correlation.Series(ticker_list[i].close, ticker_list[j].close, corr_period);

CorrPair corrPairs = new CorrPair();
						corrPairs.Correlation = corr;
						corrPairs.key = ticker_list[i].Symbol + ":" + ticker_list[j].Symbol;
						corr_list.Add(corrPairs);
						PrintDebug(corrPairs.key + "\t" + corr.Count);
					}
				}

SetContext("AAPL", false);  // A symbol with 2517 dates
				
				for (int k = corr_period + 1 ; k < Bars.Count ; k++)
				{
					// Get the data for this bar
					for (int i = 0; i < symCnt; i++)
					{
						for(int j = i+1; j < symCnt; j++)
						{
							CorrPair result = corr_list.Find(x => x.key == ticker_list[i].Symbol + ":" + ticker_list[j].Symbol);   
							PrintDebug(Bars.Date[k].ToShortDateString () + "\t" + result.key + "\t" + result.Correlation[k]);
						}
					}
				}
				
			}		
		}
	}

}
[/code]

sedelstein

3/30/2015 7:11 AM

What I'm really looking to do is after I get all the pairwise correlations for a data set (some of which have more data and some less) is to go through each day and pull out the ones on a given date where data exists.

I'm unclear on how to do it and any suggestions appreciated.

sympair1 sympair2 sympair 3
Date1 nan nan 0.95
Date2 nan nan 0.94
.
.
.
.Date n 0.93 0.86 0.93

etc.....

Thanks in advance.

sedelstein

3/30/2015 11:02 PM

Related to this as I am trying to get all pairwise correlations of the Nasdaq 100 going back in time, is it possible to synchronize two external symbols to each other rather than the primary symbol?

I am running the following over 1500 bars and the number (count) of historical correlation series changes with each symbol in the dataset