WatchLists / Evolving Watchlist
Author: nalin
Creation Date: 2/8/2010 4:39 AM
profile picture

nalin

#1
I'm trying to create the equivalent of Watchlists which were available in WL 4.

Whats the best way of doing this ?

I assumed the best way to do this was to first create a DataSet and then add or remove symbols. But the difficulty I'm having with this approach is that after I add or remove symbols the button for "Apply Symbol Changes Above to Dataset" remains disabled....so I can't save the changes.
profile picture

Eugene

#2
Install this Extension from our site: Aronow Software LLC Watchlist Static Data Provider

QUOTE:
"Apply Symbol Changes Above to Dataset" remains disabled

Some data providers don't allow modifying the DataSet after its creation.
profile picture

DaveAronow

#3
I would like to add editing of symbols but I need the Wealth-Lab .Net API to be published and Fidelity is taking their sweet time getting it done. Call them to complain (see the other postings about this).

Dave
profile picture

Eugene

#4
Without disclosing the unpublished docs re: ExtendedBehaviors (which I haven't seen anyway), here's my derivation in my own words:

1. Create a partial class that inherits from DataBehaviorUserControl. This will be your provider's tab inside the Data Manager. Here you read and write the provider settings.
CODE:
Please log in to see this code.


2. In your StaticDataProvider class, notice an optional override called ExtendedBehaviors.

Create a List<DataBehaviorUserControl> object, instantiate your partial class (step 1) and add it to the List<>.
CODE:
Please log in to see this code.


Update:

3. Wealth-Lab will use the Tag property of your UserControl for the tab's name.
profile picture

nalin

#5
The extension by Aronow Software works perfectly. Thanx !
profile picture

Sammy_G

#6
There appears to be a big limitation in the usage of this extension. Namely, you either add symbols one at a time, or select an entire dataset's symbols. So if I want to create a watchlist of say 1000 symbols from my master dataset which has 10000 symbols, I have to click 1000 times. Not my idea of fun.
profile picture

Eugene

#7
If there's any chance that those symbols sit next to each other, use the standard Windows way of selecting with Shift key: select the first symbol, press and hold the Shift key, while holding the key down click the last item.

P.S. When you asked this question in support, I wondered: what would be your solution to it? How else a GUI program should know which items user wanted to select, without having to click on them?
profile picture

Sammy_G

#8
My solution would be the way its done in v4, namely: You populate the symbol window (the one on the right) with a list of symbols you have copied from somewhere (e.g. a website), no need to be limited to selecting either one or all symbols.
profile picture

WLTrader23

#9
I almost died and went to heaven when I found your Evolving Watchlist code with the IndexHelper. Just what I need to backtest the daily changes to my fundamentals watchlist. However, I can't get it to run properly. It keeps running the Dataset and not the symbols in the DAX files. The original code debug list verifies that when run without a strategy, it does give me the proper symbols on the correct dates. However when I add any type of Position Entry code, it runs the entire associated Dataset for the complete Date Range. Below I've put in a simple crossing moving averages to make sure it wasn't my added code, but still did not work. What am doing wrong? Perhaps the placement of my the Position Entry Code?

CODE:
Please log in to see this code.
profile picture

Eugene

#10
That's right. What you removed is not the debug code. It's the symbol list representing the index composition for that bar(date). The system now isn't aware of the actual composition, and should execute trades on the complete watchlist.
profile picture

dansmo

#11
try it this way:
CODE:
Please log in to see this code.
profile picture

WLTrader23

#12
I'm in heaven now and I didn't even have to expire. Also added back in the symbolList sl = ih.WatchList( bar ) statement and it works great even using intraday. Thanks a boodle.
profile picture

WLTrader23

#13
When backtesting using the Evolving (dynamic) watchlist, is the Buy and Equity curve showing Buy and hold the equities during the period that they are in the watchlist or holding all equities for the entire date range?
profile picture

Eugene

#14
Buy & Hold doesn't close positions - it takes an equal-dollar position in each symbol of the DataSet on bar 1 and holds them till the end. Dynamic DataSet composition won't affect B&H.
profile picture

Cone

#15
Note that you can also elect to use Benchmark Buy & Hold in Preferences (F12) > Backtest Settings. Theoretically, you could create a Index symbol that that took into account a dynamic WatchList, but it would be easiest to use one like the S&P 500, Nasdaq 100, etc.
profile picture

Nadja

#16
Hello all

To start with the idea of creating dynamic watchlists, I have taken the code from "Backtesting with Dynamic (Evolving) WatchLists" and tried to implement it. Howewer, I receive the message that the index should not be out of range or negative. I have tried to change denominations and involved folders, but it remains the same.

The folder Nasdaq_100_Files contains the text files for each year, from NASDAQ_19980101.txt to NASDAQ_20120101.txt.
The folder Nasdaq_100_Set_final contains the data for all the stocks that appeared in the index during this time, equally in txt format.

Does anybody know what I overlook?

Thank you in advance for any ideas.

Kind regards

Andreas

CODE:
Please log in to see this code.
profile picture

Eugene

#17
Hello Andreas,

1. Please double check that your symbol list files follow the convention: one symbol per line, no headers, nothing else.
2. Quick debug question: would the PrintDebug() output of the "DateTime firstDateAvailable..." line be valid?
3. As the solution includes external variables such as the collection of symbol name data and the IndexHelper class, have you tried running the Strategy under debugger as shown here or here, depending on your IDE?
profile picture

Nadja

#18
Hello Eugene

Thank you for your help, I could solve the problem from above.

Now, I am much further, one step from the goal. The idea is to integrate the dynamic watchlist in a rotation strategy, the same idea as discussed in ROC Symbol Rotation - Modification of the Strategy.

In my opinion, it should be possible to remove the elements from the buy list that are not on the watchlist in the main loop, but before the entry orders. The aim is that at each bar, an up-to-date buy list is created.

The method which should do this (see below) causes a runtime error, maybe it's just something about the classes IndexHelper or List<T> that I do not fully understand.

The detailed decription is in the code.

I would really appreciate any form of help. If you need to see the whole code, it's no problem, but I didn't want to bother with it, because it's quite long!

Kind regards

Andreas


CODE:
Please log in to see this code.
profile picture

Eugene

#19
Andreas,

You should never .Remove anything from a collection you are iterating over while inside of a foreach loop. On the other hand, there are other ways to do the same thing:
CODE:
Please log in to see this code.

profile picture

Nadja

#20
Hello Eugene

Thank you for the competent answer!

With your loop, the whole code works, I just had to change line 4: if (!(sl.symbols.Contains ....)

Kind regards

Andreas
profile picture

kyokushin

#21
Hi,

Is there a chance that I get the symbol list files mentioned in "Backtesting with Dynamic (Evolving) WatchLists" for NASDAQ 100 somewhere in the internet or do I have to create them myself?

Regards

Matt
profile picture

Cone

#22
In the thread that Eugene will delete you asked...
QUOTE:
Does Norgate provide them with their historical data or do I have to create them myself?
I know that at one time Norgate was working on that product, but as far as I know, you'll still have to create those files yourself.

Here's some help for changes in the Nasdaq 100. Working backwards you should be able to create the files. (Although I should be able to write a script to do it today or tomorrow.)

Edit: Will correct and repost below..
profile picture

vk

#23
You might want to contact them and remind them, here the URL: http://www.premiumdata.net
Please don't forget to mention that you using this fantastic piece of software called Wealth-Lab. ;)
profile picture

Eugene

#24
QUOTE:
Here's some help for changes in the Nasdaq 100.

Hey, does Nasdaq 100 always have a hundred components or maybe not? Or is the supplied list of changes incorrect? I've just written a parser which shows me that the number varies. Just a couple examples:

20091221: 8 stocks replaced 7
20030102: 20 stocks replaced 19
profile picture

kyokushin

#25
I'm quite positive NASDAQ 100 always consists of 100 stocks. Perhaps the list is incorrect...
profile picture

Eugene

#26
Also, the list of changes diverges with Nasdaq-100's own sometimes, like in 2003:

http://www.nasdaq.com/indexshares/historical_data.stm

Plus, WMFI replaces a stock twice (on 20101220 and 20030102) w/o being removed itself in the middle.
profile picture

Cone

#27
We were missing this set of changes:
20081221; ADP FSLR LIFE ROST MXIM ILMN PPDI ORLY URBN JBHT WCRX replaced AMLN CDNS DISCA LAMR LEAP LVLT PETM SNDK SIRI VMED WFMI

Let's work on a corrected list and then repost.
profile picture

Cone

#28
Below is the list for the Nasdaq 100 changes. Let us know if you see any errors.

Eugene will create the text files to work with the IndexHelper. Since data for many of the symbols are no longer readily available from most data providers, we'll probably need work with the PremiumData Delisted Securities symbology. Example:

This record:
20020724; LNCR +- WCOM

Would change to:
LNCR +- WCOEQ-200404

CODE:
Please log in to see this code.
profile picture

Eugene

#29
Hi Matthias,

Have you already created the text files for IndexHelper by hand, or you're waiting for us? If you still need the files for Nasdaq-100, please create a support ticket. We'd like to keep this in private for business reasons.