Hello,
I am not sure if this is related to a problem I have been having for almost a month now, but the problem just got worse to the point where I am now forced to shut down all strategies after hours. All of my strategies are designed to run on futures during all hours. I have a thread trying to figure that out here: https://www.wealth-lab.com/Discussion/-quot-No-Updates-quot-from-Strategy-Monitor-on-IB-when-running-two-strategies-on-the-same-symbol-on-two-timeframes-10313
It was an intermittent problem that I was able to temporarily fix by restarting WL and deleting the local files for Interactive Brokers on the Data Manager. I would then find that the data would not update properly on my activated strategies, and I would restart WL and purge the data yet again. However, now IB will not update historical data beyond 6:00pm. I have tried to reset both IB and WL, along with purging IB's data and "clear internal request tracking info".
I also tried to use a streaming chart to see if it could run the strategy. It will stream the latest price, but it is missing all the data since 6:00pm, and on the chart it even believes the bar is at that time as well. I have attached a screenshot to show this strange behavior.
I noticed the problem because one of my strategies was yet again not generating the correct signals (I am cross referencing it with IQFeed). However, now that it absolutely refuses to update the data past 18:00 hours (current time is 23:20), it is inoperative. It is also strange that 18:00/6pm is the time it will not update past. This time is not significant, other than it is one hour after the futures market reopens in the evening.
When I stream the symbol on the streaming data testbed it returns the current price at the current time.
I have even tested this on a different computer and am getting the same problem. I usually use a VPS, but the same problem is persisting even on my desktop computer.
There is also unfortunately no entry into the log. WL is not noticing any issue with the data update. I have attached a couple more screenshots to show this problem is occurring no matter where I attempt to update the data.
Please help, I am completely inoperative at this time.
Thank you,
Dandude
I am not sure if this is related to a problem I have been having for almost a month now, but the problem just got worse to the point where I am now forced to shut down all strategies after hours. All of my strategies are designed to run on futures during all hours. I have a thread trying to figure that out here: https://www.wealth-lab.com/Discussion/-quot-No-Updates-quot-from-Strategy-Monitor-on-IB-when-running-two-strategies-on-the-same-symbol-on-two-timeframes-10313
It was an intermittent problem that I was able to temporarily fix by restarting WL and deleting the local files for Interactive Brokers on the Data Manager. I would then find that the data would not update properly on my activated strategies, and I would restart WL and purge the data yet again. However, now IB will not update historical data beyond 6:00pm. I have tried to reset both IB and WL, along with purging IB's data and "clear internal request tracking info".
I also tried to use a streaming chart to see if it could run the strategy. It will stream the latest price, but it is missing all the data since 6:00pm, and on the chart it even believes the bar is at that time as well. I have attached a screenshot to show this strange behavior.
I noticed the problem because one of my strategies was yet again not generating the correct signals (I am cross referencing it with IQFeed). However, now that it absolutely refuses to update the data past 18:00 hours (current time is 23:20), it is inoperative. It is also strange that 18:00/6pm is the time it will not update past. This time is not significant, other than it is one hour after the futures market reopens in the evening.
When I stream the symbol on the streaming data testbed it returns the current price at the current time.
I have even tested this on a different computer and am getting the same problem. I usually use a VPS, but the same problem is persisting even on my desktop computer.
There is also unfortunately no entry into the log. WL is not noticing any issue with the data update. I have attached a couple more screenshots to show this problem is occurring no matter where I attempt to update the data.
Please help, I am completely inoperative at this time.
Thank you,
Dandude
Rename
I had a suspicion that it may be on IB's end. I know that they do system restarts until 12am CST. I waited until midnight, and now the historical data appears to be updating correctly.
However, this is still a pretty significant issue. WL does not seem to be able to handle these data disruptions from IB and often gets stuck with incomplete data. This usually requires a WL restart to fix, but I have no warning of when it might happen. I am also hoping I don't have to get up at midnight every night to make sure this issue is not occurring.
However, this is still a pretty significant issue. WL does not seem to be able to handle these data disruptions from IB and often gets stuck with incomplete data. This usually requires a WL restart to fix, but I have no warning of when it might happen. I am also hoping I don't have to get up at midnight every night to make sure this issue is not occurring.
In your first image, it looks like data is being received in the Streaming bar after 6pm, but possibly it's the chart that isn't closing the 15-min interval. I'm on the road, but I'll take a look on Monday.
In any case, for 15 min bars and only a few instruments, I'd definitely use the Strategy Monitor with Polling. That way you don't have to rely on a continuous streaming connection for data.
Aside:
I would not use IB Streaming for trading anyway for the reasons described in the Help Guide. If you want to trade with "streaming", use the Streaming Bars in the Strategy Monitor.
In any case, for 15 min bars and only a few instruments, I'd definitely use the Strategy Monitor with Polling. That way you don't have to rely on a continuous streaming connection for data.
Aside:
I would not use IB Streaming for trading anyway for the reasons described in the Help Guide. If you want to trade with "streaming", use the Streaming Bars in the Strategy Monitor.
Ok, I hadn't considered polling because the help file says not to for IB. I will try it though.
Thanks for using the Help guide! :)
I'll make it more specific to the case we're talking about - "only a few instruments".
If you poll for dozens of symbols, it could take a long time, so generally you should not use it except for Daily bars.
I'll make it more specific to the case we're talking about - "only a few instruments".
If you poll for dozens of symbols, it could take a long time, so generally you should not use it except for Daily bars.
So once again at 6pm the data updates are failing. None of my previous tricks of restarting WL or clearing the local files is fixing the problem. I am getting the same results as described above.
Now that I have tried polling, WL is reporting that it is getting no updates as soon as it tried the first bar after 1800 (6pm). The polling method gives this output in the log:
11/19/2023 18:00:01: Status = Processing
11/19/2023 18:00:01: Pass 1 - Requesting updates for 1 symbols...
11/19/2023 18:00:02: Pass 1 - UpdateHistories returned 1 symbols
11/19/2023 18:00:02: Ran Strategy on NQZ23: 0 Signals, Run Time=489ms
11/19/2023 18:00:02: NextRun set to 11/19/2023 18:15
11/19/2023 18:00:02: Status = Completed
11/19/2023 18:00:02: Processed all Symbols
11/19/2023 18:15:00: Status = Processing
11/19/2023 18:15:00: Pass 1 - Requesting updates for 1 symbols...
11/19/2023 18:15:00: Pass 1 - UpdateHistories returned 0 symbols
11/19/2023 18:15:10: Pass 2 - Requesting updates for 1 symbols...
11/19/2023 18:15:11: Pass 2 - UpdateHistories returned 0 symbols
11/19/2023 18:15:21: Pass 3 - Requesting updates for 1 symbols...
11/19/2023 18:15:21: Pass 3 - UpdateHistories returned 0 symbols
11/19/2023 18:15:31: Pass 4 - Requesting updates for 1 symbols...
11/19/2023 18:15:31: Pass 4 - UpdateHistories returned 0 symbols
11/19/2023 18:15:41: Pass 5 - Requesting updates for 1 symbols...
11/19/2023 18:15:41: Pass 5 - UpdateHistories returned 0 symbols
11/19/2023 18:15:41: NextRun set to 11/19/2023 18:30
11/19/2023 18:15:41: Status = Incomplete
11/19/2023 18:15:41: No Updates for 1 Symbol: NQZ23
I just noticed this, and attempted a streaming bars run, but I got this output that again ends at 1800 (6pm):
11/19/2023 19:07:58: Status = Deactivated
11/19/2023 19:08:09: Status = LoadingData
11/19/2023 19:08:09: Starting Streaming Bars Thread
11/19/2023 19:08:09: Populating Data
11/19/2023 19:08:09: Calling GetHistories Pass 1
11/19/2023 19:08:10: GetHistories returned with 1 symbols
11/19/2023 19:08:10: Calling GetHistories Pass 2
11/19/2023 19:08:10: GetHistories returned with 1 symbols
11/19/2023 19:08:10: NQZ23:2,000 bars to 20231119 1800|Interactive Brokers
11/19/2023 19:08:10: Running Now
11/19/2023 19:08:10: Status = Idle
11/19/2023 19:08:10: Status = Processing
11/19/2023 19:08:11: Status = Deactivated
11/19/2023 19:08:11: Leaving Polling Thread
11/19/2023 19:08:11: Leaving Streaming Bars Thread
I am assuming it will start working again at midnight, but I don't know.
Now that I have tried polling, WL is reporting that it is getting no updates as soon as it tried the first bar after 1800 (6pm). The polling method gives this output in the log:
11/19/2023 18:00:01: Status = Processing
11/19/2023 18:00:01: Pass 1 - Requesting updates for 1 symbols...
11/19/2023 18:00:02: Pass 1 - UpdateHistories returned 1 symbols
11/19/2023 18:00:02: Ran Strategy on NQZ23: 0 Signals, Run Time=489ms
11/19/2023 18:00:02: NextRun set to 11/19/2023 18:15
11/19/2023 18:00:02: Status = Completed
11/19/2023 18:00:02: Processed all Symbols
11/19/2023 18:15:00: Status = Processing
11/19/2023 18:15:00: Pass 1 - Requesting updates for 1 symbols...
11/19/2023 18:15:00: Pass 1 - UpdateHistories returned 0 symbols
11/19/2023 18:15:10: Pass 2 - Requesting updates for 1 symbols...
11/19/2023 18:15:11: Pass 2 - UpdateHistories returned 0 symbols
11/19/2023 18:15:21: Pass 3 - Requesting updates for 1 symbols...
11/19/2023 18:15:21: Pass 3 - UpdateHistories returned 0 symbols
11/19/2023 18:15:31: Pass 4 - Requesting updates for 1 symbols...
11/19/2023 18:15:31: Pass 4 - UpdateHistories returned 0 symbols
11/19/2023 18:15:41: Pass 5 - Requesting updates for 1 symbols...
11/19/2023 18:15:41: Pass 5 - UpdateHistories returned 0 symbols
11/19/2023 18:15:41: NextRun set to 11/19/2023 18:30
11/19/2023 18:15:41: Status = Incomplete
11/19/2023 18:15:41: No Updates for 1 Symbol: NQZ23
I just noticed this, and attempted a streaming bars run, but I got this output that again ends at 1800 (6pm):
11/19/2023 19:07:58: Status = Deactivated
11/19/2023 19:08:09: Status = LoadingData
11/19/2023 19:08:09: Starting Streaming Bars Thread
11/19/2023 19:08:09: Populating Data
11/19/2023 19:08:09: Calling GetHistories Pass 1
11/19/2023 19:08:10: GetHistories returned with 1 symbols
11/19/2023 19:08:10: Calling GetHistories Pass 2
11/19/2023 19:08:10: GetHistories returned with 1 symbols
11/19/2023 19:08:10: NQZ23:2,000 bars to 20231119 1800|Interactive Brokers
11/19/2023 19:08:10: Running Now
11/19/2023 19:08:10: Status = Idle
11/19/2023 19:08:10: Status = Processing
11/19/2023 19:08:11: Status = Deactivated
11/19/2023 19:08:11: Leaving Polling Thread
11/19/2023 19:08:11: Leaving Streaming Bars Thread
I am assuming it will start working again at midnight, but I don't know.
Here is an attempt with streaming data on the strategy monitor. I figured to include it for completeness. You can see it attempted to get the data for almost ten minutes.
11/19/2023 19:28:48: Status = LoadingData
11/19/2023 19:28:48: Start Streaming Thread
11/19/2023 19:28:48: Populating Data
11/19/2023 19:28:48: Calling GetHistories Pass 1
11/19/2023 19:32:58: GetHistories returned with 1 symbols
11/19/2023 19:32:58: Calling GetHistories Pass 2
11/19/2023 19:37:32: GetHistories returned with 1 symbols
11/19/2023 19:37:32: NQZ23:2,000 bars to 20231119 1800|Interactive Brokers
11/19/2023 19:37:32: Running Now
11/19/2023 19:37:32: Status = Idle
11/19/2023 19:37:33: NQZ23 got Partial Bar from Interactive Brokers
11/19/2023 19:37:33: Status = Processing
11/19/2023 19:37:34: Ran Strategy on NQZ23: 0 Signals, Run Time=824ms
11/19/2023 19:37:34: Status = Deactivated
11/19/2023 19:37:34: Status = Completed
11/19/2023 19:37:34: Processed all Symbols
11/19/2023 19:37:34: Status = Deactivated
11/19/2023 19:37:34: Leaving Streaming Thread
11/19/2023 19:28:48: Status = LoadingData
11/19/2023 19:28:48: Start Streaming Thread
11/19/2023 19:28:48: Populating Data
11/19/2023 19:28:48: Calling GetHistories Pass 1
11/19/2023 19:32:58: GetHistories returned with 1 symbols
11/19/2023 19:32:58: Calling GetHistories Pass 2
11/19/2023 19:37:32: GetHistories returned with 1 symbols
11/19/2023 19:37:32: NQZ23:2,000 bars to 20231119 1800|Interactive Brokers
11/19/2023 19:37:32: Running Now
11/19/2023 19:37:32: Status = Idle
11/19/2023 19:37:33: NQZ23 got Partial Bar from Interactive Brokers
11/19/2023 19:37:33: Status = Processing
11/19/2023 19:37:34: Ran Strategy on NQZ23: 0 Signals, Run Time=824ms
11/19/2023 19:37:34: Status = Deactivated
11/19/2023 19:37:34: Status = Completed
11/19/2023 19:37:34: Processed all Symbols
11/19/2023 19:37:34: Status = Deactivated
11/19/2023 19:37:34: Leaving Streaming Thread
While this is not ideal for the long term, is it possible to run the strategies using IQFeed's data while placing orders on IB using symbol mapping? I just thought of that as a temporary workaround for this problem.
The answer is yes! I figured out how to use IQFeed's data since IB won't update.
Of course that will work. If you have an IQFeed subscription, you should always use that instead of IB's data.
Re: IB not updating
Like I said, I'll look at it tonight (but thanks for the reminder). It certainly should work too.
Re: IB not updating
Like I said, I'll look at it tonight (but thanks for the reminder). It certainly should work too.
Because I am trading on IB, I assumed that I should use IB data for timing/accuracy/simplicity. Are you saying to use IQFeed because it is more reliable or for any other reason(s)?
Another very practical reason is because IQFeed only allows one connection, and I prefer to be running backtests/optimizations on my much more powerful desktop while my VPS uses IB to stream data and do the trading.
IQFeed has:
1. no data limits
2. tick-by-tick accuracy for streaming
3. faster historical data requests by orders of magnitude (compared to IB)
4. and, yes, I'd definitely say is more reliable
1. no data limits
2. tick-by-tick accuracy for streaming
3. faster historical data requests by orders of magnitude (compared to IB)
4. and, yes, I'd definitely say is more reliable
Bug #1 - you noticed it too, but the Strategy Monitor fail is definitely a WL issue - it's programming the Next Run for tomorrow, not "today" when the market reopens.
I have 2 streaming charts running on CME and the last interval didn't close. We'll work on that too. Nonetheless, at 6pm ET, both IB Streaming charts closed the final bar and started building news bars without any interaction on my part when the CME re-opened -
Given that you're trading with IB, is there a reason you don't program your daily IB TWS restart after CME closes and before it reopens (and a WL restart after that)? That would "cure" everything except the last bar not closing.
Let me know how IQFeed worked for you in a streaming chart.
I have 2 streaming charts running on CME and the last interval didn't close. We'll work on that too. Nonetheless, at 6pm ET, both IB Streaming charts closed the final bar and started building news bars without any interaction on my part when the CME re-opened -
Given that you're trading with IB, is there a reason you don't program your daily IB TWS restart after CME closes and before it reopens (and a WL restart after that)? That would "cure" everything except the last bar not closing.
Let me know how IQFeed worked for you in a streaming chart.
Yes I do set the restart for after CME close and before the open. I also restart WL during this time, but it does not fix my problem.
Unfortunately, IQFeed is now not updating for me. It was working yesterday. Very frustrating.
I need to go take care of the kids but I will return soon to try to describe the problem more.
Unfortunately, IQFeed is now not updating for me. It was working yesterday. Very frustrating.
I need to go take care of the kids but I will return soon to try to describe the problem more.
After deleting my local files IQFeed appears to be updating correctly. But I am very confused as to why things keep going wrong for me.
Ah, nevermind. It is not updating the next bar now. It might work if I clear local files again, but that removes my ability to automate.
IQFeed is not streaming on the chart. The price does not move.
I would guess that this is an issue on WL's end. Although IQFeed is behaving a little differently from IB, they both appear to be stuck and not updating after hours in the same manner. WL doesn't really do anything when I ask it to update the data during this time, it just says it was completed. I even restarted WL and tried to update the Dataset with IQFeed and it did not load IQFeed before reporting the update completed. I only noticed this after I tried to open a different window using IQFeed and it then loaded the IQFeed Connection Manager.
Perhaps this is related to recent changes with WL regarding after hours/pre/post market.
Perhaps this is related to recent changes with WL regarding after hours/pre/post market.
Here is the strategy monitor's log for me trying to do a 5 minute run using IQFeed with polling, then streaming data, then streaming bars. As you can see, polling "returned 0 symbols" and missed the most recent bar. Streaming data "UpdateHistories returned 0 symbols", and it gives an error, "Status = Error". Finally streaming bars misses the last bar.
11/20/2023 18:06:48: Status = LoadingData
11/20/2023 18:06:48: Start Polling Thread
11/20/2023 18:06:48: Populating Data
11/20/2023 18:06:48: Calling GetHistories Pass 1
11/20/2023 18:06:49: GetHistories returned with 1 symbols
11/20/2023 18:06:49: Calling GetHistories Pass 2
11/20/2023 18:06:49: GetHistories returned with 1 symbols
11/20/2023 18:06:49: @NQZ23:2,000 bars to 20231120 1900|IQFeed
11/20/2023 18:06:49: Running Now
11/20/2023 18:06:49: Status = Idle
11/20/2023 18:06:49: Status = Processing
11/20/2023 18:06:49: Pass 1 - Requesting updates for 1 symbols...
11/20/2023 18:06:49: Pass 1 - UpdateHistories returned 0 symbols
11/20/2023 18:06:50: Ran Strategy on @NQZ23: 0 Signals, Run Time=782ms
11/20/2023 18:06:50: NextRun set to 11/20/2023 18:10
11/20/2023 18:06:50: Status = Completed
11/20/2023 18:06:50: Processed all Symbols
11/20/2023 18:07:00: Status = Deactivated
11/20/2023 18:07:00: Leaving Polling Thread
11/20/2023 18:07:09: Status = LoadingData
11/20/2023 18:07:09: Start Polling Thread
11/20/2023 18:07:09: Populating Data
11/20/2023 18:07:09: Calling GetHistories Pass 1
11/20/2023 18:07:09: GetHistories returned with 1 symbols
11/20/2023 18:07:09: Calling GetHistories Pass 2
11/20/2023 18:07:09: GetHistories returned with 1 symbols
11/20/2023 18:07:09: @NQZ23:2,000 bars to 20231120 1900|IQFeed
11/20/2023 18:07:09: Running Now
11/20/2023 18:07:09: Status = Idle
11/20/2023 18:07:09: Status = Processing
11/20/2023 18:07:09: Pass 1 - Requesting updates for 1 symbols...
11/20/2023 18:07:09: Pass 1 - UpdateHistories returned 0 symbols
11/20/2023 18:07:10: Ran Strategy on @NQZ23: 0 Signals, Run Time=364ms
11/20/2023 18:07:10: NextRun set to 11/20/2023 18:10
11/20/2023 18:07:10: Status = Completed
11/20/2023 18:07:10: Processed all Symbols
11/20/2023 18:07:14: Status = Deactivated
11/20/2023 18:07:19: Status = LoadingData
11/20/2023 18:07:19: Start Streaming Thread
11/20/2023 18:07:19: Populating Data
11/20/2023 18:07:19: Calling GetHistories Pass 1
11/20/2023 18:07:20: Status = Deactivated
11/20/2023 18:07:20: Leaving Polling Thread
11/20/2023 18:07:20: GetHistories returned with 1 symbols
11/20/2023 18:07:20: Calling GetHistories Pass 2
11/20/2023 18:07:20: GetHistories returned with 0 symbols
11/20/2023 18:07:20: Status = Error
11/20/2023 18:07:20: Status = Deactivated
11/20/2023 18:07:20: Leaving Streaming Thread
11/20/2023 18:07:58: Status = LoadingData
11/20/2023 18:07:58: Starting Streaming Bars Thread
11/20/2023 18:07:58: Populating Data
11/20/2023 18:07:58: Calling GetHistories Pass 1
11/20/2023 18:07:58: GetHistories returned with 1 symbols
11/20/2023 18:07:58: Calling GetHistories Pass 2
11/20/2023 18:07:58: GetHistories returned with 1 symbols
11/20/2023 18:07:58: @NQZ23:2,000 bars to 20231120 1900|IQFeed
11/20/2023 18:07:58: Running Now
11/20/2023 18:07:58: Status = Idle
11/20/2023 18:07:58: Status = Processing
11/20/2023 18:08:00: Ran Strategy on @NQZ23: 0 Signals, Run Time=361ms
11/20/2023 18:08:00: Status = Deactivated
11/20/2023 18:08:00: Status = Completed
11/20/2023 18:08:00: Processed all Symbols
11/20/2023 18:08:00: Status = Deactivated
11/20/2023 18:08:00: Leaving Streaming Bars Thread
11/20/2023 18:06:48: Status = LoadingData
11/20/2023 18:06:48: Start Polling Thread
11/20/2023 18:06:48: Populating Data
11/20/2023 18:06:48: Calling GetHistories Pass 1
11/20/2023 18:06:49: GetHistories returned with 1 symbols
11/20/2023 18:06:49: Calling GetHistories Pass 2
11/20/2023 18:06:49: GetHistories returned with 1 symbols
11/20/2023 18:06:49: @NQZ23:2,000 bars to 20231120 1900|IQFeed
11/20/2023 18:06:49: Running Now
11/20/2023 18:06:49: Status = Idle
11/20/2023 18:06:49: Status = Processing
11/20/2023 18:06:49: Pass 1 - Requesting updates for 1 symbols...
11/20/2023 18:06:49: Pass 1 - UpdateHistories returned 0 symbols
11/20/2023 18:06:50: Ran Strategy on @NQZ23: 0 Signals, Run Time=782ms
11/20/2023 18:06:50: NextRun set to 11/20/2023 18:10
11/20/2023 18:06:50: Status = Completed
11/20/2023 18:06:50: Processed all Symbols
11/20/2023 18:07:00: Status = Deactivated
11/20/2023 18:07:00: Leaving Polling Thread
11/20/2023 18:07:09: Status = LoadingData
11/20/2023 18:07:09: Start Polling Thread
11/20/2023 18:07:09: Populating Data
11/20/2023 18:07:09: Calling GetHistories Pass 1
11/20/2023 18:07:09: GetHistories returned with 1 symbols
11/20/2023 18:07:09: Calling GetHistories Pass 2
11/20/2023 18:07:09: GetHistories returned with 1 symbols
11/20/2023 18:07:09: @NQZ23:2,000 bars to 20231120 1900|IQFeed
11/20/2023 18:07:09: Running Now
11/20/2023 18:07:09: Status = Idle
11/20/2023 18:07:09: Status = Processing
11/20/2023 18:07:09: Pass 1 - Requesting updates for 1 symbols...
11/20/2023 18:07:09: Pass 1 - UpdateHistories returned 0 symbols
11/20/2023 18:07:10: Ran Strategy on @NQZ23: 0 Signals, Run Time=364ms
11/20/2023 18:07:10: NextRun set to 11/20/2023 18:10
11/20/2023 18:07:10: Status = Completed
11/20/2023 18:07:10: Processed all Symbols
11/20/2023 18:07:14: Status = Deactivated
11/20/2023 18:07:19: Status = LoadingData
11/20/2023 18:07:19: Start Streaming Thread
11/20/2023 18:07:19: Populating Data
11/20/2023 18:07:19: Calling GetHistories Pass 1
11/20/2023 18:07:20: Status = Deactivated
11/20/2023 18:07:20: Leaving Polling Thread
11/20/2023 18:07:20: GetHistories returned with 1 symbols
11/20/2023 18:07:20: Calling GetHistories Pass 2
11/20/2023 18:07:20: GetHistories returned with 0 symbols
11/20/2023 18:07:20: Status = Error
11/20/2023 18:07:20: Status = Deactivated
11/20/2023 18:07:20: Leaving Streaming Thread
11/20/2023 18:07:58: Status = LoadingData
11/20/2023 18:07:58: Starting Streaming Bars Thread
11/20/2023 18:07:58: Populating Data
11/20/2023 18:07:58: Calling GetHistories Pass 1
11/20/2023 18:07:58: GetHistories returned with 1 symbols
11/20/2023 18:07:58: Calling GetHistories Pass 2
11/20/2023 18:07:58: GetHistories returned with 1 symbols
11/20/2023 18:07:58: @NQZ23:2,000 bars to 20231120 1900|IQFeed
11/20/2023 18:07:58: Running Now
11/20/2023 18:07:58: Status = Idle
11/20/2023 18:07:58: Status = Processing
11/20/2023 18:08:00: Ran Strategy on @NQZ23: 0 Signals, Run Time=361ms
11/20/2023 18:08:00: Status = Deactivated
11/20/2023 18:08:00: Status = Completed
11/20/2023 18:08:00: Processed all Symbols
11/20/2023 18:08:00: Status = Deactivated
11/20/2023 18:08:00: Leaving Streaming Bars Thread
I'm not sure if this is related but I have discovered some other strange behavior while looking for a workaround with the strategy monitor. I changed my system's time to UTC timezone hoping that telling WL that it was the next day would allow it to update correctly. This did not work, and I found that when I activate the strategies, they appear to choose an almost random time far in the future for the "next run". Usually, of course, the next run should be on the next bar, for my strategies, in the next 5 or 15 minutes. But instead it sets the next run to some time two to five hours in the future!
Here is a screenshot:
Here is a screenshot:
Next Run is in Local time... so that would be in the future for you if you put the machine in UTC or Greenwich.
You're throwing a lot at me all at once, so let's just start from the beginning.
Just look at the first chart about 3 or 4 posts ago. You're requesting NQZ23. That's not an IQ Feed symbol. IB is returning the history data but you selected IQFeed to Stream (which would work if you did an IQFeed streaming symbol mapping to from NQZ23 to @NQZ23).
Another thing is that you should know that IQFeed only returns data in ET - you need to create a different "CME Eastern" market when using IQFeed.
1. In Markets & Symbols, copy the CME market and modify it for Eastern. Name it CME Eastern.
2. Assign @NQ### to CME Eastern in Markets and Symbols (and enter the contract specs.)
3. Use @NQZ23 for historical and streaming
4. Select IB in the Order Manager and click the Symbol Mapping button.
Assign @NQZ23=NQZ23, @NQH24=NQH24, etc. as required for trading.
Note:
With that setup the Strategy is running with @NQZ23 but the symbol traded is NQZ23.
Currently, "Use Live Positions" isn't going to work with the symbol mapping scenario.
You're throwing a lot at me all at once, so let's just start from the beginning.
Just look at the first chart about 3 or 4 posts ago. You're requesting NQZ23. That's not an IQ Feed symbol. IB is returning the history data but you selected IQFeed to Stream (which would work if you did an IQFeed streaming symbol mapping to from NQZ23 to @NQZ23).
Another thing is that you should know that IQFeed only returns data in ET - you need to create a different "CME Eastern" market when using IQFeed.
1. In Markets & Symbols, copy the CME market and modify it for Eastern. Name it CME Eastern.
2. Assign @NQ### to CME Eastern in Markets and Symbols (and enter the contract specs.)
3. Use @NQZ23 for historical and streaming
4. Select IB in the Order Manager and click the Symbol Mapping button.
Assign @NQZ23=NQZ23, @NQH24=NQH24, etc. as required for trading.
Note:
With that setup the Strategy is running with @NQZ23 but the symbol traded is NQZ23.
Currently, "Use Live Positions" isn't going to work with the symbol mapping scenario.
Ok sorry. I was wondering if I should share everything I was finding on here and thought maybe it would be helpful.
I have never done that with the markets & symbols, so I will have to try that.
You're right on the chart above, but when I am using the strategy monitor with IQFeed I put in the IQFeed symbol @NQZ23. Last night I could not get IQFeed to update.
I have never done that with the markets & symbols, so I will have to try that.
You're right on the chart above, but when I am using the strategy monitor with IQFeed I put in the IQFeed symbol @NQZ23. Last night I could not get IQFeed to update.
Well the beginning is that IB data is not updating from 6pm-12am and I have no solution for it. I would prefer to use IB to IQFeed because I use IQFeed on another machine for backtesting and optimization.
I will attempt the IQFeed fix you mentioned and let you know my results.
I will attempt the IQFeed fix you mentioned and let you know my results.
The Strategy Monitor Next Run for markets that open and close on different calendar days will be fixed later this week in Build 61.
As for the the failure between 6pm and 12am, I can't duplicate that one.
As for the the failure between 6pm and 12am, I can't duplicate that one.
The markets & symbols fix did not work unfortunately. IQFeed is not updating, neither the strategy monitor nor the streaming chart are working for me.
Should I schedule a call to fix this? Since you cannot duplicate the problem. I have been working on these strategies since I joined WL 10 months ago without any problem and now I cannot function.
Should I schedule a call to fix this? Since you cannot duplicate the problem. I have been working on these strategies since I joined WL 10 months ago without any problem and now I cannot function.
Sure, I'll have a call with you dan, it's working at my end as well so let's see if we can figure out what's going wrong. Write us at support@wealth-lab.com.
Thanks Glitch. I just realized a possible reason that IQFeed is behaving so poorly for me. When I signed up for IQFeed, it was for historical data, and therefore I only subscribed to the "core service". I think this may be giving me delayed data. I will ask IQFeed customer service for clarification on this before we do a call.
This perhaps gives more urgency to figuring out the IB data failure. I would like to avoid paying an additional $126/mo for the CME live data (I believe that may be the package I would need).
Glitch or Cone, do you have an IB account/data subscription? That is the data feed update failure I would really like to fix. Have either of you been able to recreate the issues I am having with IB data? That is very consistently failing each evening after 6pm CST and then works perfectly after 12am CST. If/when we do a call, that would probably be what I am looking to troubleshoot. I would imagine we would need to do the call after 6pm on a weekday so I could show you what is happening on my end.
This perhaps gives more urgency to figuring out the IB data failure. I would like to avoid paying an additional $126/mo for the CME live data (I believe that may be the package I would need).
Glitch or Cone, do you have an IB account/data subscription? That is the data feed update failure I would really like to fix. Have either of you been able to recreate the issues I am having with IB data? That is very consistently failing each evening after 6pm CST and then works perfectly after 12am CST. If/when we do a call, that would probably be what I am looking to troubleshoot. I would imagine we would need to do the call after 6pm on a weekday so I could show you what is happening on my end.
QUOTE:For IQFeed, you need special subscriptions for futures and other products that are not U.S. Stocks.
I think this may be giving me delayed data.
Of course I have accounts and multiple data subscriptions with IB. For your IB data, log in to your Account > Settings > Market Data Subscriptions and make sure you're subscribed to "US Securities Snapshot and Futures Value Bundle - Trader Workstation".
Yes I have that subscription. IB had been working up until a few days ago, although with a few weird hiccups for the last month or so, if you look at my original post from October 23rd. So it was working for the last nine months until these problems showed up.
Is anyone else able to get IB to update CME contracts between 6pm and 12am CST? Even more specifically, can anyone get NQZ23 to update using the strategy monitor during this time?
I am still getting the same results where it stops at 6pm and gives signals only to that point. Unfortunately build 61 did not fix the problem.
I am still getting the same results where it stops at 6pm and gives signals only to that point. Unfortunately build 61 did not fix the problem.
I'm confident you're the only one having trouble here, so let's look at your set up - starting with IB TWS and Local Time Zones.
Do they match?
Do they match?
Yes. Both are set to my time zone, CST. I even tried to change both to UTC last week and it did not change the behavior (In fact made it worse, which I documented above).
I have reviewed and checked IB's setup based on the Help files. I have uninstalled and reinstalled WL.
I have attached a screenshot of my Time zone settings, IB API settings, WL IB settings, trading, and data preferences.
I have reviewed and checked IB's setup based on the Help files. I have uninstalled and reinstalled WL.
I have attached a screenshot of my Time zone settings, IB API settings, WL IB settings, trading, and data preferences.
Nothing's jumping out at me there.
Can you record a short video of WealthLab - full app showing status bars - from about 5:59:55pm ET to 6:01:05 ET?
I'll do the same tonight.
Can you record a short video of WealthLab - full app showing status bars - from about 5:59:55pm ET to 6:01:05 ET?
I'll do the same tonight.
Sure I will try to do that this evening. However I have not recorded my screen before, do you recommend a program?
Also, which status bars are you referring to? What parts of WL should I make sure are visible?
Also, you mentioned ET, but the problem occurs at 6pm CST, is that what you meant?
Also, which status bars are you referring to? What parts of WL should I make sure are visible?
Also, you mentioned ET, but the problem occurs at 6pm CST, is that what you meant?
Best to just make the whole application visible.
QUOTE:I use Camtasia, but probably the free version of https://www.bandicam.com/ will work for you.
do you recommend a program?
QUOTE:The entire application. Don't cut anything off of the charts. If you use the Strategy Monitor, show the Log too with the Strategy selected.
Also, which status bars are you referring to? What parts of WL should I make sure are visible?
QUOTE:Well, no.
Also, you mentioned ET, but the problem occurs at 6pm CST, is that what you meant?
NQ is traded on the CME (total hours) from 17:00 CST to 16:00 CST the next day. That's 18:00 EST to 17:00 EST.
The CME should already be trading NQ for an hour by 6pm CST - it opens at 5pm CST.
That gives me another idea of something to look at - Market & Symbols. Here's my record for NQ. Does yours match it?
The only difference is the "margin" field, which I wouldn't imagine would affect anything.
Yes, so my system actually updates just fine for the first hour of afterhours trading from 5pm-6pm, but then at exactly 6pm it stops updating the data. So the problem does not occur at the evening open but actually one hour later. Also I should mention that at 6pm it will not appear to have any issue, but at the next update you will see it will fail to update, at 6:05pm for 5 minute bars, for example.
Yes, so my system actually updates just fine for the first hour of afterhours trading from 5pm-6pm, but then at exactly 6pm it stops updating the data. So the problem does not occur at the evening open but actually one hour later. Also I should mention that at 6pm it will not appear to have any issue, but at the next update you will see it will fail to update, at 6:05pm for 5 minute bars, for example.
I had misunderstood and had expected the problem to occur right when the market re-opened.
Okay, then.. I'll look for it an hour later this time!
.. that's midnight for me :/ ... interesting, midnight GMT... hmm.
Okay, then.. I'll look for it an hour later this time!
.. that's midnight for me :/ ... interesting, midnight GMT... hmm.
Haha yes I was wondering if perhaps the 00:00 UTC was significant.
I was going to say then you could be up for the WL zoom call, but that's tomorrow!
I will hold off on making a video recording to see if you can reproduce the problem.
I was going to say then you could be up for the WL zoom call, but that's tomorrow!
I will hold off on making a video recording to see if you can reproduce the problem.
The problem is confirmed.
I've been looking at it for 45 minutes and I'm not going to be able to sleep now!
I've been looking at it for 45 minutes and I'm not going to be able to sleep now!
Uh oh... I know that feeling!
This appears to have been an unfortunate side effect of fixing a DST time zone issue. I think we've got it fixed for Build 62, and I'll make sure with a test tonight.
Sorry for the trouble it caused, and thanks for bringing it to our attention.
Sorry for the trouble it caused, and thanks for bringing it to our attention.
I'm glad you figured it out. As a side note I would vote 100 times against DST!
Thanks for your diligence in hanging with me as we troubleshot. I was able to adapt to the issue while it existed so it didn't cause me any major problems.
Hope you got some sleep!
Thanks for your diligence in hanging with me as we troubleshot. I was able to adapt to the issue while it existed so it didn't cause me any major problems.
Hope you got some sleep!
Your Response
Post
Edit Post
Login is required