Using-Transaction-Weight-is-running-slow-in-my-C-code

If there's a problem with speed, the most probable cause are Event Providers, which are notoriously slow to update.
Uncheck all Event Providers that you're not using for the backtest.

(And there's not enough code there to determine anything.)

But how is the weight TimeSeries is defined and populated?

Here is the full code.

Thank you,
Larry

CODE:
using WealthLab.Backtest;
using System;
using WealthLab.Core;
using WealthLab.Indicators;
using System.Drawing;
using System.Collections.Generic;

namespace WealthScript3 
{
   public class Rotation : UserStrategyBase
   {
      public Rotation()
      {
       

         //Daily bar settings 
         AddParameter("ROC Period", ParameterType.Int32, 150, 30, 300, 10);  // Default = 3    
         AddParameter("SMA Period", ParameterType.Int32, 190, 30, 300, 10); // Default = 10   
         AddParameter("n Symbols", ParameterType.Int32, 5, 2, 10, 1);    // Default = 3      

         AddParameter("Stop Loss", ParameterType.Double, 10, 2, 30, 2);

      }

      ROC _avgROC;
      SMA _smaBenchmark;
      BarHistory _bmBars;

      string _seriesKey = "Average ROC";
      string _bmSymbol = "^GSPC";

      //the list of symbols that we should buy each bar
      private static List<BarHistory> buys = new List<BarHistory>();

      //create the weight indicator and stash it into the BarHistory object for reference in PreExecute
      public override void Initialize(BarHistory bars)
      {
         StartIndex = Math.Max(Parameters[0].AsInt, Parameters[1].AsInt);

         _avgROC = ROC.Series(bars.Close, Parameters[0].AsInt);
         bars.Cache[_seriesKey] = _avgROC;

         _bmBars = GetHistory(bars, _bmSymbol, "Major U.S. and world indices");
         _smaBenchmark = SMA.Series(_bmBars.Close, Parameters[1].AsInt);

      }

      public override void PreExecute(DateTime dt, List<BarHistory> participants)
      {
         //store the symbols' _avgROC value in their BarHistory instances
         foreach (BarHistory bh in participants)
         {
            TimeSeries symbolRoc = (TimeSeries)bh.Cache[_seriesKey];
            int idx = GetCurrentIndex(bh);
            double rocVal = symbolRoc[idx];

            if (idx <= symbolRoc.FirstValidIndex)
               rocVal = -1.0e6;

            bh.UserData = rocVal; //save the current _avgROC value along with the BarHistory instance
         }

         //sort the participants by _avgROC value (highest to lowest)
         participants.Sort((a, b) => b.UserDataAsDouble.CompareTo(a.UserDataAsDouble));

         //keep the top 3 symbols
         buys.Clear();

         //market filter
         int currentBar = GetCurrentIndex(_bmBars);
         if (_bmBars.Close[currentBar] > _smaBenchmark[currentBar])
         {
            for (int n = 0; n < Parameters[2].AsInt; n++)
            {
               if (n >= participants.Count)
                  break;
               buys.Add(participants[n]);
               int idx = GetCurrentIndex(participants[n]);
               DateTime dte = participants[n].DateTimes[idx];
            }
         }

         foreach (Position pos in OpenPositionsAllSymbols)
         {
            if (!buys.Contains(pos.Bars))
            {
               // find the relevant bar for the given current date
               int idx = pos.Bars.IndexOf(dt, true);
               if (idx >= 0 && pos.Bars.DateTimes[idx].IsLastTradingDayOfMonth(pos.Bars))
               {
                  ClosePosition(pos, OrderType.Market);
               }
               Backtester.CancelationCode = 220;
               value = 1.0 - (Parameters[3].AsDouble / 100.0);
               //ClosePosition(foundPosition0, OrderType.Stop, foundPosition0.EntryPrice * value, "Sell at stop loss");
               ClosePosition(pos, OrderType.Stop, pos.EntryPrice * value, "Sell at stop loss");
            }
         }

         foreach (BarHistory bh in buys)
         {
            if (!HasOpenPosition(bh, PositionType.Long))
            {
               int idx = GetCurrentIndex(bh);
               bool isFirstTradingDayOfMonth = idx > 0 ? bh.DateTimes[idx - 1].IsLastTradingDayOfMonth(bh) : false;
               if (isFirstTradingDayOfMonth) // this code makes buy/sell signals only on 1st day of the month, so I don't get daily trade signals repeating during the month
               {
                  //Transaction t = PlaceTrade(bh, TransactionType.Buy, OrderType.Market);
                  _transaction = PlaceTrade(bh, TransactionType.Buy, OrderType.Market, 0, 0, "Buy At Market (1)");
                  _transaction.Weight = weight[idx] * 1;
               }
            }
         }
      }
      public override void Execute(BarHistory bars, int idx)
      {
         weight = new RSI(bars.Close,5);
      }
      
      private double value;
      private bool trailing;
      private IndicatorBase weight;
      private Transaction _transaction;

   }
   
}

OK I see, this is a modification of paul1986's code inherited from here: https://www.wealth-lab.com/Discussion/Monthly-strategy-to-give-daily-signals-on-1st-trading-day-of-month-only-9189

You define the weight as a class level variable, populate it without caching (hence the slowness) in Execute() and expect it to return a correct value in PreExecute (which runs before Execute by design)? I doubt that even the RSI values would be correct (i.e. belong to the desired BarHistory) or even populated.

I think you could prefill and cache the series in Initialize much like the original code is doing to the average ROC...

CODE:
_avgROC = ROC.Series(bars.Close, Parameters[0].AsInt);
bars.Cache[_seriesKey] = _avgROC;

...and then take the weight value for the BarHistory from the cache later.

From the point of view of the compiler, when you "box" a datatype, you should always do the unboxing with that same datatype. In the code below, _avgROC is defined as a "ROC" datatype (which is okay), but that means you need to unbox it as an "ROC" and not a "TimeSeries".

CODE:
      ROC _avgROC; //declared as an indicator, not a TimeSeries
      ...
      _avgROC = ROC.Series(bars.Close, Parameters[0].AsInt);
      bars.Cache[_seriesKey] = _avgROC;  // "boxed" as ROC, not TimeSeries
      ...
      ROC symbolRoc = (ROC)bh.Cache[_seriesKey];  // correct
      //TimeSeries symbolRoc = (TimeSeries)bh.Cache[_seriesKey];  // VERY DANGEROUS

Remember, the compiler has no idea what the original datatype was at the unboxing point (unless it's a virtual datatype, which is not the case here). Honestly, I'm surprise the original code works at all because an indicator (like ROC, which is based on IndicatorBase) has other "stuff" in it that's not present in a TimeSeries type.

Another option would be to declare _avgROC as a TimeSeries from the start (since you're not using the other "stuff" contained in an indicator type). That would save you a little memory.

Eugene wrote:

"I think you could prefill and cache the series in Initialize much like the original code is doing to the average ROC..."

with example:

CODE:
_avgROC = ROC.Series(bars.Close, Parameters[0].AsInt);
bars.Cache[_seriesKey] = _avgROC;

Larry wrote:

I originally changed this code to add transaction weight:

CODE:
//Transaction t = PlaceTrade(bh, TransactionType.Buy, OrderType.Market);
                  _transaction = PlaceTrade(bh, TransactionType.Buy, OrderType.Market, 0, 0, "Buy At Market (1)");
                  _transaction.Weight = weight[idx] * 1;

which is slowing code speed, way much....

Larry also made these changes from superticker:

CODE:
 ROC _avgROC; //declared as an indicator, not a TimeSeries
      ...
      _avgROC = ROC.Series(bars.Close, Parameters[0].AsInt);
      bars.Cache[_seriesKey] = _avgROC;  // "boxed" as ROC, not TimeSeries
      ...
      ROC symbolRoc = (ROC)bh.Cache[_seriesKey];  // correct
      //TimeSeries symbolRoc = (TimeSeries)bh.Cache[_seriesKey];  // VERY DANGEROUS

Can you please be more specific in what I am supposed to change.

Here is my latest code:

CODE:
using WealthLab.Backtest;
using System;
using WealthLab.Core;
using WealthLab.Indicators;
using System.Drawing;
using System.Collections.Generic;

namespace WealthScript3 
{
   public class Rotation : UserStrategyBase
   {
      public Rotation()
      {
         // Monthly bar settings
         //AddParameter("ROC Period", ParameterType.Int32, 150, 1, 10, 1);  // Default = 3      
         //AddParameter("SMA Period", ParameterType.Int32, 190, 3, 15, 1); // Default = 10   
         //AddParameter("n Symbols", ParameterType.Int32, 5, 2, 5, 1);    // Default = 3       

         //Daily bar settings 
         AddParameter("ROC Period", ParameterType.Int32, 150, 30, 300, 10);  // Default = 3    
         AddParameter("SMA Period", ParameterType.Int32, 190, 30, 300, 10); // Default = 10   
         AddParameter("n Symbols", ParameterType.Int32, 5, 2, 10, 1);    // Default = 3      

         AddParameter("Stop Loss", ParameterType.Double, 10, 2, 30, 2);

      }

      ROC _avgROC; //declared as an indicator, not a TimeSeries
      SMA _smaBenchmark;
      BarHistory _bmBars;

      string _seriesKey = "Average ROC";
      string _bmSymbol = "^GSPC";

      //the list of symbols that we should buy each bar
      private static List<BarHistory> buys = new List<BarHistory>();

      //create the weight indicator and stash it into the BarHistory object for reference in PreExecute
      public override void Initialize(BarHistory bars)
      {
         StartIndex = Math.Max(Parameters[0].AsInt, Parameters[1].AsInt);

         _avgROC = ROC.Series(bars.Close, Parameters[0].AsInt);
         bars.Cache[_seriesKey] = _avgROC; // "boxed" as ROC, not TimeSeries

         _bmBars = GetHistory(bars, _bmSymbol, "Major U.S. and world indices");
         _smaBenchmark = SMA.Series(_bmBars.Close, Parameters[1].AsInt);

      }

      public override void PreExecute(DateTime dt, List<BarHistory> participants)
      {
         //store the symbols' _avgROC value in their BarHistory instances
         foreach (BarHistory bh in participants)
         {
         //   TimeSeries symbolRoc = (TimeSeries)bh.Cache[_seriesKey];  // VERY DANGEROUS
             ROC symbolRoc = (ROC)bh.Cache[_seriesKey];  // correct
            
            int idx = GetCurrentIndex(bh);
            double rocVal = symbolRoc[idx];

            if (idx <= symbolRoc.FirstValidIndex)
               rocVal = -1.0e6;

            bh.UserData = rocVal; //save the current _avgROC value along with the BarHistory instance
         }

         //sort the participants by _avgROC value (highest to lowest)
         participants.Sort((a, b) => b.UserDataAsDouble.CompareTo(a.UserDataAsDouble));

         //keep the top 3 symbols
         buys.Clear();

         //market filter
         int currentBar = GetCurrentIndex(_bmBars);
         if (_bmBars.Close[currentBar] > _smaBenchmark[currentBar])
         {
            for (int n = 0; n < Parameters[2].AsInt; n++)
            {
               if (n >= participants.Count)
                  break;
               buys.Add(participants[n]);
               int idx = GetCurrentIndex(participants[n]);
               DateTime dte = participants[n].DateTimes[idx];
            }
         }

         foreach (Position pos in OpenPositionsAllSymbols)
         {
            if (!buys.Contains(pos.Bars))
            {
               // find the relevant bar for the given current date
               int idx = pos.Bars.IndexOf(dt, true);
               if (idx >= 0 && pos.Bars.DateTimes[idx].IsLastTradingDayOfMonth(pos.Bars))
               {
                  ClosePosition(pos, OrderType.Market);
               }
               Backtester.CancelationCode = 220;
               value = 1.0 - (Parameters[3].AsDouble / 100.0);
               //ClosePosition(foundPosition0, OrderType.Stop, foundPosition0.EntryPrice * value, "Sell at stop loss");
               ClosePosition(pos, OrderType.Stop, pos.EntryPrice * value, "Sell at stop loss");
            }
         }

         foreach (BarHistory bh in buys)
         {
            if (!HasOpenPosition(bh, PositionType.Long))
            {
               int idx = GetCurrentIndex(bh);
               bool isFirstTradingDayOfMonth = idx > 0 ? bh.DateTimes[idx - 1].IsLastTradingDayOfMonth(bh) : false;
               if (isFirstTradingDayOfMonth) // this code makes buy/sell signals only on 1st day of the month, so I don't get daily trade signals repeating during the month
               {
                  //Transaction t = PlaceTrade(bh, TransactionType.Buy, OrderType.Market);
                  _transaction = PlaceTrade(bh, TransactionType.Buy, OrderType.Market, 0, 0, "Buy At Market (1)");
                  _transaction.Weight = weight[idx] * 1;
               }
            }
         }
      }
      public override void Execute(BarHistory bars, int idx)
      {
         weight = new RSI(bars.Close,5);
      }
      
      private double value;
      private bool trailing;
      private IndicatorBase weight;
      private Transaction _transaction;

   }
   
}

Thank you,
Larry

You are recreating the weight value on EVERY bar! Don't do that. Remove

CODE:
weight = new RSI(bars.Close, 5);

from the Execute method and put that statement into the Initialize method. The speed difference is enormous.

All Reply# 4 is saying is to cache the weight values so you're not recreating them on each Execute() loop. And that's all the code below is doing, nothing more. As to whether this code actually works or not, I don't know. I haven't tried it.

CODE:
   public class Rotation : UserStrategyBase
   {
      ...
      RSI weight;
      
      public override void Initialize(BarHistory bars)
      {
         ...
         weight = new RSI(bars.Close, 5);
         bars.Cache["_weight"] = weight;
         ...
      }

      public override void PreExecute(DateTime dt, List<BarHistory> participants)
      {
          ...
         foreach (BarHistory bh in buys)
         {
            if (!HasOpenPosition(bh, PositionType.Long))
            {
               int idx = GetCurrentIndex(bh);
               bool isFirstTradingDayOfMonth = idx > 0 ? bh.DateTimes[idx - 1].IsLastTradingDayOfMonth(bh) : false;
               if (isFirstTradingDayOfMonth) // this code makes buy/sell signals only on 1st day of the month, so I don't get daily trade signals repeating during the month
               {
                  Transaction transaction = PlaceTrade(bh, TransactionType.Buy, OrderType.Market, 0, 0, "Buy At Market (1)");
                  RSI weight = (RSI)bh.Cache["_weight"];
                  transaction.Weight = weight[idx];
               }
            }
         }
      }

      public override void Execute(BarHistory bars, int idx) { }
   }

Perhaps this is obvious, but we have two separate RSI weight indicators declared here. One is associated with the Rotation class and the other is associated with the PreExecute-if block. One has nothing to do with the other.

I have a question. Could the RSI indicator that's declared in the Rotation class be instead declared in Initialize as a local variable? My thinking is once the reference (pointer) to that indicator is cached, the garbage collector will still recognize it even if its local existence (within Initialize) disappears. But I could be wrong.

Eugene, Paul and Superticker,

The speed difference is insane. Like lightening!!! Went down from multiple minutes to seconds... Huge improvement.

Thank you so much.

Larry