Same strategy, same data... DIFFERENT RESULTS?
Author: TheInvis
Creation Date: 8/13/2008 9:25 PM
profile picture

TheInvis

#1
Hmmm. Don't know what's going on, but after I run a Strategy, if I hit GO without changing any settings, WL5.0 delivers a different performance result. Again and again...

!?!?!

The results are "somewhat consistent": A loss of $24,000, a loss of $31,000, a loss of $46,000 vs. a solid, unchanging result on the B&H side. The dividends also seem to stay the same from one run to another.

I suspect it may be the 10% position size, which I was using in this case, rather than the $5000 RP which I had been using. (I hadn't noticed any inconsistencies with the $5000 RP setting.)


ps. Just went back and tried and this doesn't seem to be a problem in RP mode. Same performance result to the penny for 3 clicks, using the exact same Strategy, Watchlist and settings (other than RP mode change) as before.
profile picture

Eugene

#2
Don't worry, everything goes as planned.

What you're facing is the result of a modification to the randomization algorithm for position priority that differs Wealth-Lab Developer/Pro .NET from its previous versions.

You were running a Strategy with position priority not assigned in Strategy code. In this case, a random value for priority is automatically assigned. In Multi-Symbol Backtest mode, your Strategy haven't had enough capital to execute all generated alerts - this explains why new and different result sets are experienced with each MSB run. In legacy versions, position priority was pseudo-random while now it's truly random.

As the User Guide tells us:
QUOTE:
Consequently, when trades are rejected and priorities are random, the results of successive but identical backtests will most often not match - even when using the same settings.


Solution: assign position priority.
profile picture

Cone

#3
Also, rememeber the dialog that you clicked "OK" to?...

-----------------------
Warning
-----------------------
123 trades were not included in teh backtest results due to insufficient simulated capital. Use Raw Profit mode to ensure all trades are always executed. You can find the number of traes not included at the bottom of the Trades list.

[] Don't show this message again.
-----------------------
OK
-----------------------

In a nutshell, if you don't assign your own Position.Priority in the Strategy code, and trades are not executed, you will get random results in the simulation. Every result is just as valid as the next as a result of choosing different trades for the available trading capital.
profile picture

kfmfe04

#4
Would be nice if this were an option:

CheckBox: Use Random Seed _______ for psuedo-random.

Where you check on and enter a value (for repeatable tests: in Eugene's terms, "pseudo-random")

If the check is off, then it runs "truly random" in Eugene's terms.

---------------------------------------------

Being able to reproduce results is useful for working through each trade in detail so we can understand exactly what a particular system is doing.


profile picture

Eugene

#5
QUOTE:
Being able to reproduce results is useful for working through each trade in detail so we can understand exactly what a particular system is doing.


Assigning a Position priority does just that, but keeps the application interface clean and streamlined ;-)
profile picture

kfmfe04

#6
QUOTE:
Assigning a Position priority does just that, but keeps the application interface clean and streamlined ;-)


Thanks for the hint!

I found the documentation describing how to do that - will try it out...