- ago
I'm trying to understand how to interpret the results when running a Monte Carlo test using Basic Run mode on a limit-order system that includes NSF positions.

In the original backtest, I have:

~1140 actual positions

~70 NSF positions

However, after running the Monte Carlo test with Basic Run mode for multiple iterations, many of the simulations end up with only a few hundred positions (200-600, no NSF), which is significantly fewer than the original backtest.

From the documentation, my understanding is that Basic Run mode repeatedly re-runs the same backtest logic, but with a randomized sampling of the original trades.

My question is whether the following interpretation is correct:
- On a given day in the original backtest, there may be enough raw entry signals to fill (for example) 10 slots.
- In a Monte Carlo iteration, because only a subset of trades is randomly sampled, it’s possible that far fewer than 10 of those trades are present.
- As a result, many slots go unfilled in that iteration, which cascades into a much lower total number of positions over the full simulation.

Is this expected behavior in Basic Run mode, and is this the primary reason why some Monte Carlo simulations end up with far fewer positions than the original backtest? Thanks.
0
232
4 Replies

Reply

Bookmark

Sort
Cone8
 ( 16.64% )
- ago
#1
The Basic Run just runs the Strategy with the same settings many times. You'd only see a difference if 1) you have NSF positions, and, 2) the Strategy does not assign Transaction.Weight.

I can only guess that the big difference in number of Positions must be explained by something else random in the Strategy. I guess we'd have to see the Strategy to determine that without guessing.
0
johnbens8
 ( 0.00% )
- ago
#2
I believe I encountered the same problem today. I am conducting a strategy test, and when I manually click "Run Backtest“ for many times, the number of trades is usually around 4200.

However, when I use "Basic Run" for simulation, the number of trades varies greatly, even reaching a minimum of less than 100 trades.

I am not sure if this phenomenon is related to the order type used, as I am using a Stop Order for entry.

I have not observed this phenomenon in other strategies that use Market Order for entry.
0
- ago
#3
Here is an example strategy that generates far fewer signals in MC test. Run it on WD Nasdaq100 for last ten years with 10% equity, 10 positions, 1.1 margin factor, I have ~1670 positions and 185 NFS positions. With basic run in MC the position counts are from ~40-240. From equity curve of those MC runs I can see that there's no open positions for most of the time.

CODE:
using WealthLab.Backtest; using System; using WealthLab.Core; using WealthLab.Data; using WealthLab.Indicators; using System.Collections.Generic; using System.Linq; namespace WealthScript1 {    public class MyStrategy : UserStrategyBase    {       // Buy list - symbols that pass entry conditions       private static List<BarHistory> buys = new List<BarHistory>();       // Maximum number of entry signals       private const int maxEntrySignals = 10;       public MyStrategy() : base()       {          StartIndex = 200;       }       public override void Initialize(BarHistory bars)       {          // Create indicators          rsi = new RSI(bars.Close, 5);          PlotIndicator(rsi, new WLColor(0, 0, 0));          _startIndexList.Add(rsi);          qqqHistory = GetHistory(bars, "QQQ");          qqqClose = qqqHistory.Close;          qqqSMA200 = new SMA(qqqHistory.Close, 200);          PlotIndicator(qqqSMA200, new WLColor(0, 0, 255), default, default, "QQQPane");          _startIndexList.Add(qqqSMA200);          barsLow = bars.Low;          percentMultiplier = 1.00;          percentMultiplier = (100.0 - percentMultiplier) / 100.0;          buyLimitPrice = barsLow * percentMultiplier;          PlotStopsAndLimits(3);          // Cache indicators for PreExecute access          bars.Cache["rsi"] = rsi;          bars.Cache["qqqClose"] = qqqClose;          bars.Cache["qqqSMA200"] = qqqSMA200;          bars.Cache["buyLimitPrice"] = buyLimitPrice;          bars.Cache["qqqHistory"] = qqqHistory;          foreach (IndicatorBase ib in _startIndexList)             if (ib.FirstValidIndex > StartIndex)                StartIndex = ib.FirstValidIndex;       }       public override void PreExecute(DateTime dt, List<BarHistory> participants)       {          // Clear previous buy list          buys.Clear();          // Evaluate each participant for entry conditions          foreach (BarHistory participant in participants)          {             // Check if this participant passes all entry conditions             string reason = "";             bool passed = ShouldEnterPosition(participant, out reason);             if (passed)             {                // Store RSI value for ranking                int idx = GetCurrentIndex(participant);                IndicatorBase rsiInd = (IndicatorBase)participant.Cache["rsi"];                double rsiValue = rsiInd[idx];                participant.Cache["rsiValue"] = rsiValue;                buys.Add(participant);             }          }          // Sort by RSI value (ascending - lowest RSI first) and limit to maxEntrySignals          if (buys.Count > maxEntrySignals)          {             buys = buys.OrderBy(b => (double)b.Cache["rsiValue"]).Take(maxEntrySignals).ToList();          }       }       private bool ShouldEnterPosition(BarHistory bars, out string reason)       {          int idx = GetCurrentIndex(bars);          reason = "";          // Get indicators from cache          IndicatorBase rsiInd = (IndicatorBase)bars.Cache["rsi"];          TimeSeries qqqClose = (TimeSeries)bars.Cache["qqqClose"];          IndicatorBase qqqSMA200 = (IndicatorBase)bars.Cache["qqqSMA200"];          // Condition 1: RSI < 25.00          double rsiValue = rsiInd[idx];          if (rsiValue >= 25.00)          {             reason = $"RSI={rsiValue:F2} >= 25.00";             return false;          }          // Condition 2: QQQ Close > QQQ SMA200          double qqqCloseValue = qqqClose[idx];          double qqqSMA200Value = qqqSMA200[idx];          if (qqqCloseValue <= qqqSMA200Value)          {             reason = $"QQQ Close={qqqCloseValue:F2} <= QQQ SMA200={qqqSMA200Value:F2}";             return false;          }          return true;       }       public override void Execute(BarHistory bars, int idx)       {          Position foundPosition0 = FindOpenPosition(0);          bool inBuyList = buys.Contains(bars);          if (foundPosition0 == null)          {             // Entry logic - buy if symbol is in the buy list             if (inBuyList)             {                // Get buy limit price from cache                TimeSeries buyLimitPrice = (TimeSeries)bars.Cache["buyLimitPrice"];                double limitPrice = buyLimitPrice[idx];                Backtester.CancelationCode = 1;                _transaction = PlaceTrade(bars, TransactionType.Buy, OrderType.Limit, limitPrice, 0, "Buy " + 1.00 + "% below " + barsLow.Description);             }          }          else          {             // Exit conditions             // Exit: Close above previous high             if (idx > 0 && bars.Close[idx] > bars.High[idx - 1])             {                Backtester.CancelationCode = 1;                ClosePosition(foundPosition0, OrderType.Market, 0, "Sell At Market - Price Above Previous High");             }          }       }       private IndicatorBase rsi;       private TimeSeries qqqClose;       private IndicatorBase qqqSMA200;       private BarHistory qqqHistory;       private double percentMultiplier;       private TimeSeries barsLow;       private TimeSeries buyLimitPrice;       private Transaction _transaction;       private List<IndicatorBase> _startIndexList = new List<IndicatorBase>();    } }

0
johnbens8
 ( 0.00% )
- ago
#4
I use building blocks to construct strategies. Today, I did a simple test and found that the problem may be in Symbol Ranking by Indicator.

When I add this block to the strategy, the number of results in the MC Basic Run becomes abnormal. I feel that this block itself does not produce "randomness," and this phenomenon seems like a bug.
0

Reply

Bookmark

Sort