Programatically setting primary DataSeries
Author: walkerjoe
Creation Date: 6/9/2010 4:45 AM
profile picture

walkerjoe

#1
Is there a way to programatically set the primary symbol for the Bars object.

Trying to create a strategy that works with multiple symbols from a watchlist one after the other (and writes the results into a file) - Running on the whole watchlist does not work (Memory issues - intraday data for a longer period is used).

SetContext() etc does not work for my strategy as I don't want to have extra bars created when those are missing in one symbol and I need to sync secondary series to that symbol.

Many thanks
profile picture

Eugene

#2
For writing results to the file in a Strategy that loops by DataSetSymbols, you can disable synchronization:
CODE:
Please log in to see this code.

As a result, no extra bars will be appended.
profile picture

walkerjoe

#3
Guess I have to provide a bit more detail:

1. Bars object is set by manual selection from watchlist / DataSet with symbol to start the strategy. This is what I want to automate or to be able to replace this primary series / Bars object. If I can start the strategy for a specific symbol e.g. via parameter passing when calling the strategy that would most likely also solve the problem.

2. SetContext to secondary symbol

3. load external series

4. sync external series to secondary symbol

If I sync the external series these are automatically synchronized to the primary series and the results for the secondary symbol become different. I've also realized that I have to reset the context to the secondary symbol every time after changing the DateScale / RestoreScale() is used.

The problem with the extra bars is less with the appending but more with interpolation - creating bars when missing in between.
Example - 1 minute series-1 has data for every minute trading time. Series-2 is missing data e.g. for minutes 9:32 and 9:34
data is then added for these minutes while there was actually no trade at that time.
profile picture

Eugene

#4
Let's look at it from a different angle. Maybe a design pattern or an optinal solution already exists. Can you exemplify this -
QUOTE:
Trying to create a strategy that works with multiple symbols from a watchlist one after the other (and writes the results into a file)
profile picture

walkerjoe

#5
The problem is that WL is running out of memory when running the strategy on the whole dataset.

So I'm trying to run it for one symbol at a time - that works so far.

To do this programatically I currently can only think of the following solutions:
(1) being able to start a strategy for a specific symbol programatically
(2) scripting WL so that each symbol in the dataset is clicked with the mouse one after the other (too slow and potentially error prone)
(2) doing the whole strategy calculations outside of WL and then load the results (would be very disappointing)

The key issue I believe here is that when working with multiple data series those need to be synchronized otherwise results become kind of random. I so far have nevertheless only found a way to sync to the primary series i.e. the one the symbol was clicked to start the strategy. What I need is to sync to the then secondary series / the actually processed series so that no false trading times are created.

If I can run a strategy for a specific dataset, symbol and start it from another strategy / program then this certainly solves the problem.

I have not tried if loading the data for the secondary series from file (LoadFromFile) into the primary Bars object and then reset the symbol. Could this be a possibility to replace the data instead of changing the context?
profile picture

Eugene

#6
QUOTE:
The problem is that WL is running out of memory when running the strategy on the whole dataset.

How much memory is installed on your computer? Are you working with the x64 version of WLD? What kind of DataSet do you have (data scale, data loading range, number of symbols, approximate number of series)?
profile picture

walkerjoe

#7
Memory is plenty (4GB) but working with the 32bit version on Windows XP so it can't be used above 3GB. I have no 64bit Windows installed (Unix / Linux only)

working with 1 minute data for 1 year range, symbols up 1000 (currently only 400 used - above 100 WL goes bye-bye) - 5 external series in the moment.

Issue is that the strategy is loading / (mem) caching all series at once and then tries to execute the strategy. Results from that run are written to files and then used in follow up calculations (other strategies).

While loading of all series in a dataset certainly makes a lot of sense for most strategies here it's the killer. If it does not fall over during that part it falls over while executing the strategy.

profile picture

Cone

#8
Exactly right. More memory addressing is required, so you'll need a 64-bit computer/OS with a 64-bit version of Wealth-Lab. As a rule of thumb for 32-bit Wealth-Lab, you can run an MSB for 100 symbols over 10 months at the 1-minute scale.

That's about 80,000 bars per symbol, or 8M bars total. On 64-bit and using less than 8G of memory, I've run 5-year MSBs for 100 1-minute symbols.
profile picture

Eugene

#9
Installing a 64-bit Windows version, and using the 64-bit WLD 5.6 is the most straightforward option, as opposed to inventing workarounds. Caching is not an issue, it's by design and is required for performance visualizers. You're already pushing the envelope with 400 symbols, it's more than your installed memory could hold even on a 64-bit O/S. See for yourself:
CODE:
Please log in to see this code.

Required RAM = 400 * (1440 * 250) * 6 * 8 = 6.7 Gb

On a 32-bit OS, WL5's memory usage is topping out at 1.7GB of virtual memory, subsequently throwing an "OutOfMemoryException".
This website uses cookies to improve your experience. We'll assume you're ok with that, but you can opt-out if you wish (Read more).