I'm trying to understand how to interpret the results when running a Monte Carlo test using Basic Run mode on a limit-order system that includes NSF positions.
In the original backtest, I have:
~1140 actual positions
~70 NSF positions
However, after running the Monte Carlo test with Basic Run mode for multiple iterations, many of the simulations end up with only a few hundred positions (200-600, no NSF), which is significantly fewer than the original backtest.
From the documentation, my understanding is that Basic Run mode repeatedly re-runs the same backtest logic, but with a randomized sampling of the original trades.
My question is whether the following interpretation is correct:
- On a given day in the original backtest, there may be enough raw entry signals to fill (for example) 10 slots.
- In a Monte Carlo iteration, because only a subset of trades is randomly sampled, it’s possible that far fewer than 10 of those trades are present.
- As a result, many slots go unfilled in that iteration, which cascades into a much lower total number of positions over the full simulation.
Is this expected behavior in Basic Run mode, and is this the primary reason why some Monte Carlo simulations end up with far fewer positions than the original backtest? Thanks.
In the original backtest, I have:
~1140 actual positions
~70 NSF positions
However, after running the Monte Carlo test with Basic Run mode for multiple iterations, many of the simulations end up with only a few hundred positions (200-600, no NSF), which is significantly fewer than the original backtest.
From the documentation, my understanding is that Basic Run mode repeatedly re-runs the same backtest logic, but with a randomized sampling of the original trades.
My question is whether the following interpretation is correct:
- On a given day in the original backtest, there may be enough raw entry signals to fill (for example) 10 slots.
- In a Monte Carlo iteration, because only a subset of trades is randomly sampled, it’s possible that far fewer than 10 of those trades are present.
- As a result, many slots go unfilled in that iteration, which cascades into a much lower total number of positions over the full simulation.
Is this expected behavior in Basic Run mode, and is this the primary reason why some Monte Carlo simulations end up with far fewer positions than the original backtest? Thanks.
Rename
The Basic Run just runs the Strategy with the same settings many times. You'd only see a difference if 1) you have NSF positions, and, 2) the Strategy does not assign Transaction.Weight.
I can only guess that the big difference in number of Positions must be explained by something else random in the Strategy. I guess we'd have to see the Strategy to determine that without guessing.
I can only guess that the big difference in number of Positions must be explained by something else random in the Strategy. I guess we'd have to see the Strategy to determine that without guessing.
I believe I encountered the same problem today. I am conducting a strategy test, and when I manually click "Run Backtest“ for many times, the number of trades is usually around 4200.
However, when I use "Basic Run" for simulation, the number of trades varies greatly, even reaching a minimum of less than 100 trades.
I am not sure if this phenomenon is related to the order type used, as I am using a Stop Order for entry.
I have not observed this phenomenon in other strategies that use Market Order for entry.
However, when I use "Basic Run" for simulation, the number of trades varies greatly, even reaching a minimum of less than 100 trades.
I am not sure if this phenomenon is related to the order type used, as I am using a Stop Order for entry.
I have not observed this phenomenon in other strategies that use Market Order for entry.
Here is an example strategy that generates far fewer signals in MC test. Run it on WD Nasdaq100 for last ten years with 10% equity, 10 positions, 1.1 margin factor, I have ~1670 positions and 185 NFS positions. With basic run in MC the position counts are from ~40-240. From equity curve of those MC runs I can see that there's no open positions for most of the time.
CODE:
using WealthLab.Backtest; using System; using WealthLab.Core; using WealthLab.Data; using WealthLab.Indicators; using System.Collections.Generic; using System.Linq; namespace WealthScript1 { public class MyStrategy : UserStrategyBase { // Buy list - symbols that pass entry conditions private static List<BarHistory> buys = new List<BarHistory>(); // Maximum number of entry signals private const int maxEntrySignals = 10; public MyStrategy() : base() { StartIndex = 200; } public override void Initialize(BarHistory bars) { // Create indicators rsi = new RSI(bars.Close, 5); PlotIndicator(rsi, new WLColor(0, 0, 0)); _startIndexList.Add(rsi); qqqHistory = GetHistory(bars, "QQQ"); qqqClose = qqqHistory.Close; qqqSMA200 = new SMA(qqqHistory.Close, 200); PlotIndicator(qqqSMA200, new WLColor(0, 0, 255), default, default, "QQQPane"); _startIndexList.Add(qqqSMA200); barsLow = bars.Low; percentMultiplier = 1.00; percentMultiplier = (100.0 - percentMultiplier) / 100.0; buyLimitPrice = barsLow * percentMultiplier; PlotStopsAndLimits(3); // Cache indicators for PreExecute access bars.Cache["rsi"] = rsi; bars.Cache["qqqClose"] = qqqClose; bars.Cache["qqqSMA200"] = qqqSMA200; bars.Cache["buyLimitPrice"] = buyLimitPrice; bars.Cache["qqqHistory"] = qqqHistory; foreach (IndicatorBase ib in _startIndexList) if (ib.FirstValidIndex > StartIndex) StartIndex = ib.FirstValidIndex; } public override void PreExecute(DateTime dt, List<BarHistory> participants) { // Clear previous buy list buys.Clear(); // Evaluate each participant for entry conditions foreach (BarHistory participant in participants) { // Check if this participant passes all entry conditions string reason = ""; bool passed = ShouldEnterPosition(participant, out reason); if (passed) { // Store RSI value for ranking int idx = GetCurrentIndex(participant); IndicatorBase rsiInd = (IndicatorBase)participant.Cache["rsi"]; double rsiValue = rsiInd[idx]; participant.Cache["rsiValue"] = rsiValue; buys.Add(participant); } } // Sort by RSI value (ascending - lowest RSI first) and limit to maxEntrySignals if (buys.Count > maxEntrySignals) { buys = buys.OrderBy(b => (double)b.Cache["rsiValue"]).Take(maxEntrySignals).ToList(); } } private bool ShouldEnterPosition(BarHistory bars, out string reason) { int idx = GetCurrentIndex(bars); reason = ""; // Get indicators from cache IndicatorBase rsiInd = (IndicatorBase)bars.Cache["rsi"]; TimeSeries qqqClose = (TimeSeries)bars.Cache["qqqClose"]; IndicatorBase qqqSMA200 = (IndicatorBase)bars.Cache["qqqSMA200"]; // Condition 1: RSI < 25.00 double rsiValue = rsiInd[idx]; if (rsiValue >= 25.00) { reason = $"RSI={rsiValue:F2} >= 25.00"; return false; } // Condition 2: QQQ Close > QQQ SMA200 double qqqCloseValue = qqqClose[idx]; double qqqSMA200Value = qqqSMA200[idx]; if (qqqCloseValue <= qqqSMA200Value) { reason = $"QQQ Close={qqqCloseValue:F2} <= QQQ SMA200={qqqSMA200Value:F2}"; return false; } return true; } public override void Execute(BarHistory bars, int idx) { Position foundPosition0 = FindOpenPosition(0); bool inBuyList = buys.Contains(bars); if (foundPosition0 == null) { // Entry logic - buy if symbol is in the buy list if (inBuyList) { // Get buy limit price from cache TimeSeries buyLimitPrice = (TimeSeries)bars.Cache["buyLimitPrice"]; double limitPrice = buyLimitPrice[idx]; Backtester.CancelationCode = 1; _transaction = PlaceTrade(bars, TransactionType.Buy, OrderType.Limit, limitPrice, 0, "Buy " + 1.00 + "% below " + barsLow.Description); } } else { // Exit conditions // Exit: Close above previous high if (idx > 0 && bars.Close[idx] > bars.High[idx - 1]) { Backtester.CancelationCode = 1; ClosePosition(foundPosition0, OrderType.Market, 0, "Sell At Market - Price Above Previous High"); } } } private IndicatorBase rsi; private TimeSeries qqqClose; private IndicatorBase qqqSMA200; private BarHistory qqqHistory; private double percentMultiplier; private TimeSeries barsLow; private TimeSeries buyLimitPrice; private Transaction _transaction; private List<IndicatorBase> _startIndexList = new List<IndicatorBase>(); } }
I use building blocks to construct strategies. Today, I did a simple test and found that the problem may be in Symbol Ranking by Indicator.
When I add this block to the strategy, the number of results in the MC Basic Run becomes abnormal. I feel that this block itself does not produce "randomness," and this phenomenon seems like a bug.
When I add this block to the strategy, the number of results in the MC Basic Run becomes abnormal. I feel that this block itself does not produce "randomness," and this phenomenon seems like a bug.
Your Response
Post
Edit Post
Login is required