Same system with the same DataSet produces different performance results
Author: mikesblack
Creation Date: 5/22/2016 6:19 PM
profile picture

mikesblack

#1
I made a copy of my system and the data set that I use on my laptop to my main computer. With position options selected using the same parameters on 1500 bars my main computer shows 6.37 percent return and 2403 trades. My laptop, all things the same produces a 8.79 percent return and 4139 trades.

When I run on all positions, my laptop produced 24,320 trades and 60.93 percent return; however on my main computer the same system produced 12,541 trades with a 45.39 percent gain.

I was careful to make sure everything was the same from system to system and position options as well as starting capitol. Why is there such a wide disparity between the two systems?

In fact, I tested different systems( same system for each computer) , same data set, same position options and still finding very different results.
profile picture

Eugene

#2
FAQ > Every time I run a Strategy I get a different result. What am I missing?

Even then if you follow the FAQ's advice (which I believe should do it), the two computers may still have different...

a) sets of data - unless physically synchronized from one PC to another
b) PosSizer library version AND any other applicable extension versions
c) Preferred Values AND/OR Strategy Parameters
d) system revisions - unless physically synchronized from one PC to another as XML files
e) Preferences (Commissions etc)
f) You have enough things to check by now.

Same strategy, same data... DIFFERENT RESULTS?
profile picture

mikesblack

#3
The advice from the reading is to use Position priority pos sizer. I tried it without too many changes. I ran the systems in raw profit mode. Laptop: 5682 trades Desktop: 3020 trades.
On a. sets of data, I have the same symbols. If it is a data issue, which data set can I trust?
b. Same
c. unclear
d. Different versions of windows. Why should this matter?
e STRAPS are the same for the systems tested.

So, I guess the pressing concern is which PC to trust and how I can verify that. The trade number difference is huge and so I am concerned about discovering whats happening. If you wish for me to create a ticket, please let me know.
profile picture

mikesblack

#4
Even on individual stocks. I tested two. Both have the same data amount of bars. My laptop produced nearly twice the number of trades. Raw profit mode.
profile picture

Eugene

#5
QUOTE:
The advice from the reading is to use Position priority pos sizer.

Of course not. Never does it state this. I suggest you revisit the FAQ.

QUOTE:
c. unclear

c) Both are explained in the User Guide.

QUOTE:
d. Different versions of windows. Why should this matter?

I can not help but agree with you. It's about strategy revision - see the remark on XML file which should make it pretty clear.

QUOTE:
e STRAPS are the same for the systems tested.

This question was about the Preferences.

QUOTE:
The trade number difference is huge and so I am concerned about discovering whats happening. If you wish for me to create a ticket, please let me know.

No, I don't want you to create a ticket because this ain't no Wealth-Lab issue. There's no miracle: something is different across your PCs and your quest now is to figure out what exactly.

1. Synchronize your strategy via XML files (User Guide > Data > Where data are stored).
2. Synchronize your data via *.WL files. As an alternative, inspect the data symbol by symbol.
3. Synchronize your WL settings between the PCs by copying the files (How to transfer Strategies and other data between computers?). If this sounds complicated to you then triple check the Preferences and all settings including addons - Market Manager in the first place if using any data provider other than Fidelity
4. Update all extensions

I hope this should give enough hints to keep you busy :)
profile picture

Cone

#6
Another Preference settings that can cause this disparity between computers:
Preferences (F12) > Backtesting Settings > Use Worst Trades in Portfolio Simulation. (I suspect this is selected on your main computer.)

To a lesser extent, Turn off limit/stop order rounding entirely can affect results too. (This option should generally be enabled for stock simulations that span a number of years in order to use more than 2-decimal pricing.)
profile picture

mikesblack

#7
Hi Eugene, I'll look at the items you have written. It might be difficult implementing as I am unfamiliar with these items.
Hi Robert, I have indeed not selected "Turn off limit/stop order rounding or turned on "Use Worst Trades in Portfolio Simulation". I have a third computer to do my testing.
12541 trades and 45.39% on my school laptop(new pc) vs Home PC 12464 trades and 44.5%.

In addition, testing another system shows 85.8 % and 29589 trades on school laptop and Home PC shows 26602 trades and 76.72 % .
It seems that my school laptop and my home PC, while slightly different results are more alike than my other laptop I tested on yesterday. The results from my regular laptop seems way off in comparison.

I appreciate your support, but I currently feel confused on the steps Eugene has suggessted. Can I send you my system for you to either Eugene and or Robert to test on? I would appreciate keeping my system private, so can I send it through a support ticket please?
profile picture

Eugene

#8
Still no to support tickets as yesterday. Mike, you've got access to software but tutoring, coding, and private troubleshooting are not covered by the support deal. Our duty is to give you advice and explicit directions but actually following them is up to you. I realize that finding the difference in your data, strategy or preferences on 2 PCs is not the sexiest thing in the world but you are in the best position to do it.

1. Have you already checked out the FAQ in my first reply? The simplest first step suggested by it would be to activate "Use Worst Trades in Portfolio Simulation" on both computers which should keep Wealth-Lab from randomizing your trades.

2. I'll be surprised if Step 1 won't change things, and then you have to follow my advice in post #3 but don't feel confused: it's simply a matter of enabling hidden file visibility and copying a directory. Anyone can do it: How to transfer Strategies and other data between computers?
profile picture

mikesblack

#9
Eugene, I'll test "use worst trades in portfolio simulations" and update you accordingly. Why won't copying and pasting code into a new strategy window work? I don't understand how things could change by including a system I copied from an email with code. So, please explain why one would transfer a strategy using the method described in the manual rather than just do as I did, cut and paste?
profile picture

mikesblack

#10
Using worst trades, the results show over twice the number of trades on my laptop. I'm not sure I can fix it right now as I am confused to start with why there would be this kind of disparity. I understand your role as regards private tutoring, etc., but this issue seems beyond the scope of just having one of you guys help me write a system, etc. There seems to be something wrong, but I am stuck to change it. I don't know how to solve this mystery. Thank you for your help so far.
profile picture

Eugene

#11
Copying and pasting code into a new strategy window should work provided that every dependency is identical - like version of applicable extensions, WLP, all preferences, data files (physically copied or shared over the network), et cetera. See posts #2 and #6 for the complete list.

Only you're making this a mystery. Just follow our advices step by step. Please realize that nobody else in the whole world is prepared to make troubleshooting on your remote PCs better than you are. Sure this is going to a tedious work but you've got all the necessary instructions above. The ball is in your court now.
profile picture

mikesblack

#12
Thank you for your input Eugene.