Optimization Net Profit Different than Backtest Net Profit
Author: ronc
Creation Date: 9/19/2015 3:46 PM
profile picture

ronc

#1
I often optimize strategies for maximum Net Profit using Particle Swarm Optimization, single symbol, and raw profit mode. I then sort the optimization results on Net Profit, click the top row (max Net Profit), right click, and select "Assign Preferred Values to the Selected Symbol for this Row". Then I click the PV button to use preferred values. Then I look at the Performance Tab, where Net Profit for all trades equals the Net Profit that was shown in the selected row in the optimization results. Today, for the first time, I observed that after doing this the Net Profit for all trades in the Performance Tab is different from that in the optimization results. I confirmed that the PVs are in use by comparing the values in the Strategy Parameter window with those in the optimization results. I have never seen this before and am stumped. The only difference in what I am doing now is that I now have a very large number of optimization parameters (about 30). I know the large number of opt parameters is not good, and I will reduce that number, but I am wondering if there is some more fundamental issue going on here. I am reluctant to past all of my code here as it is long.
profile picture

Eugene

#2
Either your strategy must set Position.Priority properly or "Use worst trades in Port. Sim." must be enabled in Wealth-Lab's Preferences > Backtest settings. Here's a possible cause:

Every time I run a Strategy I get a different result. What am I missing?
profile picture

ronc

#3
"Use Worst Trades in Portfolio Simulation" is set. Also, I am using RP mode.

I do not get different results for identical backtests. Each backtest with the same settings yields the same result. The problem is that the backtest does not produce the same result as the optimizer, when using the preferred values selected from the best optimizer row. In the screen shots below you can see that the optimizer's best row produced Net Profit = 5,200.09. Using those PVs in a backtest produced Net Profit = 102.80.



profile picture

ronc

#4
I think I tracked this down to a bug in my code in in strategy parameter CreateParameter() statements. Upon fixing those the issue is gone.
profile picture

Eugene

#5
Have you made sure that all of those 30 parameters have unique description strings?

EDIT: Glad you seem to have figured this out yourself :)
profile picture

Panache

#6
I know this is an old thread, but I ran into a similar problem, but with a different cause. Class level variables are not reset during optimizations, so make sure they are reset within Execute().

CODE:
Please log in to see this code.
causes problems with optimization

CODE:
Please log in to see this code.
works as expected.