- ago
Hi, this is actually a huge issue I'm having and makes any strategy I'm trying to use inconsistent.

It appears that while strategy refreshes the run, often enough limit order doesn't get refreshed. Every time I noticed it doesn't, there's a "Status - Incomplete" in log file.




I am using:

- 60 minute bars (intraday)
- Streaming data
- KuCoin streaming provider

I thought the problem maybe could be in zero-volume bars, but after checking LYXE-USDT on kucoin, there obviously was volume every hour. What seems to be the problem here?



0
523
Solved
14 Replies

Reply

Bookmark

Sort
Cone8
 ( 25.05% )
- ago
#1
Are there any KuCoin messages in the Log Viewer (Ctrl + L)?
What type of data update were you using? Polling? Streaming?
0
- ago
#2
I will provide you with KuCoin message log if there's any once I get home in a few hours, I was using Streaming data with 60 minute candlesticks.

I can provide you the strategy I am using so you can maybe test out yourself if you can replicate the problem.
0
Cone8
 ( 25.05% )
- ago
#3
The strategy is not required. It’s clear that data for bar wasn’t received/compiled. Since it worked sometimes, we can at least discard the possibility of a time zone/sync issue.

I suspect you’ll find in the log that there were streaming disconnects. For 60 minute bars, you should use polling. For a single symbol, I’d use polling for all intervals, even 1 min.
0
- ago
#4
I assumed that changing streaming to polling was going to work, which I changed before going to work, but the problem is even worse now, as I get zero limit orders.



there is nothing in log viewer that would indicate anything regarding kuCoin, there's a timesync issue at 07:16 today but I synchronized the clock 8 minutes later:



these are the settings:



There is one thing that I DID do though, is that at 07:07 I started the "run now" function because it left out the limit order in previous hour, which did return me and placed a limit order. At the same time, I started POLLING instead of STREAMING DATA. After that, no limit orders were created. Could this have done anything to mess with the algorithm? What about the market clocks? for example, if you put EUREX in trading strategy settings and Cryptos for the software?



also this is the strategy, theoretically it should cancel out the last limit order and place a new one every hour, is that correct? (TrendStrengthB is ALWAYS above that treshold, so that is a non-issue).


08/05/2023 07:07:16: Running Now
08/05/2023 07:07:17: Status = Processing
08/05/2023 07:07:18: Ran Strategy on LYXE-USDT: 1 Signals, Run Time=670ms
08/05/2023 07:07:18: Status = Deactivated
08/05/2023 07:07:18: Status = Completed
08/05/2023 07:07:18: Processed all Symbols
08/05/2023 07:07:18: Status = Deactivated
08/05/2023 07:07:18: Leaving Streaming Thread
08/05/2023 07:07:22: Status = LoadingData
08/05/2023 07:07:22: Start Streaming Thread
08/05/2023 07:07:22: Populating Data
08/05/2023 07:07:22: Calling GetHistories Pass 1
08/05/2023 07:07:22: GetHistories returned with 1 symbols
08/05/2023 07:07:22: Calling GetHistories Pass 2
08/05/2023 07:07:23: GetHistories returned with 1 symbols
08/05/2023 07:07:23: LYXE-USDT:8.007 bars to 20230508 0500|KuCoin
08/05/2023 07:07:23: NextRun set to 08/05/2023 08:00
08/05/2023 07:07:23: Status = Idle
08/05/2023 07:07:24: LYXE-USDT got Partial Bar from KuCoin
08/05/2023 07:07:26: Status = Deactivated
08/05/2023 07:07:27: Leaving Streaming Thread
08/05/2023 07:07:31: Status = LoadingData
08/05/2023 07:07:31: Start Polling Thread
08/05/2023 07:07:31: Populating Data
08/05/2023 07:07:31: Calling GetHistories Pass 1
08/05/2023 07:07:32: GetHistories returned with 1 symbols
08/05/2023 07:07:32: Calling GetHistories Pass 2
08/05/2023 07:07:32: GetHistories returned with 1 symbols
08/05/2023 07:07:32: LYXE-USDT:8.007 bars to 20230508 0500|KuCoin
08/05/2023 07:07:32: NextRun set to 08/05/2023 08:00
08/05/2023 07:07:32: Status = Idle
08/05/2023 07:07:34: Running Now
08/05/2023 07:07:36: Status = Processing
08/05/2023 07:07:36: Pass 1 - Requesting updates for 1 symbols...
08/05/2023 07:07:36: Pass 1 - UpdateHistories returned 0 symbols
08/05/2023 07:07:37: Ran Strategy on LYXE-USDT: 1 Signals, Run Time=667ms
08/05/2023 07:07:37: NextRun set to 08/05/2023 08:00
08/05/2023 07:07:37: Status = Completed
08/05/2023 07:07:37: Processed all Symbols
08/05/2023 07:07:47: Status = Deactivated
08/05/2023 07:07:47: Leaving Polling Thread
08/05/2023 07:07:52: Status = LoadingData
08/05/2023 07:07:52: Start Polling Thread
08/05/2023 07:07:52: Populating Data
08/05/2023 07:07:52: Calling GetHistories Pass 1
08/05/2023 07:07:53: GetHistories returned with 1 symbols
08/05/2023 07:07:53: Calling GetHistories Pass 2
08/05/2023 07:07:53: GetHistories returned with 1 symbols
08/05/2023 07:07:53: LYXE-USDT:8.007 bars to 20230508 0500|KuCoin
08/05/2023 07:07:53: NextRun set to 08/05/2023 08:00
08/05/2023 07:07:53: Status = Idle
08/05/2023 08:00:00: Status = Processing
08/05/2023 08:00:00: Pass 1 - Requesting updates for 1 symbols...
08/05/2023 08:00:00: Pass 1 - UpdateHistories returned 0 symbols
08/05/2023 08:00:10: Pass 2 - Requesting updates for 1 symbols...
08/05/2023 08:00:10: Pass 2 - UpdateHistories returned 0 symbols
08/05/2023 08:00:20: Pass 3 - Requesting updates for 1 symbols...
08/05/2023 08:00:20: Pass 3 - UpdateHistories returned 0 symbols
08/05/2023 08:00:30: Pass 4 - Requesting updates for 1 symbols...
08/05/2023 08:00:30: Pass 4 - UpdateHistories returned 0 symbols
08/05/2023 08:00:40: Pass 5 - Requesting updates for 1 symbols...
08/05/2023 08:00:40: Pass 5 - UpdateHistories returned 0 symbols
08/05/2023 08:00:40: NextRun set to 08/05/2023 09:00
08/05/2023 08:00:40: Status = Incomplete
08/05/2023 08:00:40: No Updates for 1 Symbol: LYXE-USDT
08/05/2023 09:00:00: Status = Processing
08/05/2023 09:00:00: Pass 1 - Requesting updates for 1 symbols...
08/05/2023 09:00:00: Pass 1 - UpdateHistories returned 0 symbols
08/05/2023 09:00:10: Pass 2 - Requesting updates for 1 symbols...
08/05/2023 09:00:10: Pass 2 - UpdateHistories returned 0 symbols
08/05/2023 09:00:20: Pass 3 - Requesting updates for 1 symbols...
08/05/2023 09:00:20: Pass 3 - UpdateHistories returned 0 symbols
08/05/2023 09:00:30: Pass 4 - Requesting updates for 1 symbols...
08/05/2023 09:00:30: Pass 4 - UpdateHistories returned 0 symbols
08/05/2023 09:00:40: Pass 5 - Requesting updates for 1 symbols...
08/05/2023 09:00:40: Pass 5 - UpdateHistories returned 0 symbols
08/05/2023 09:00:40: NextRun set to 08/05/2023 10:00
08/05/2023 09:00:40: Status = Incomplete
08/05/2023 09:00:40: No Updates for 1 Symbol: LYXE-USDT
08/05/2023 10:00:01: Status = Processing
08/05/2023 10:00:01: Pass 1 - Requesting updates for 1 symbols...
08/05/2023 10:00:01: Pass 1 - UpdateHistories returned 0 symbols
08/05/2023 10:00:11: Pass 2 - Requesting updates for 1 symbols...
08/05/2023 10:00:11: Pass 2 - UpdateHistories returned 0 symbols
08/05/2023 10:00:21: Pass 3 - Requesting updates for 1 symbols...
08/05/2023 10:00:21: Pass 3 - UpdateHistories returned 0 symbols
08/05/2023 10:00:31: Pass 4 - Requesting updates for 1 symbols...
08/05/2023 10:00:31: Pass 4 - UpdateHistories returned 0 symbols
08/05/2023 10:00:41: Pass 5 - Requesting updates for 1 symbols...
08/05/2023 10:00:41: Pass 5 - UpdateHistories returned 0 symbols
08/05/2023 10:00:41: NextRun set to 08/05/2023 11:00
08/05/2023 10:00:41: Status = Incomplete
08/05/2023 10:00:41: No Updates for 1 Symbol: LYXE-USDT
08/05/2023 11:00:01: Status = Processing
08/05/2023 11:00:01: Pass 1 - Requesting updates for 1 symbols...
08/05/2023 11:00:01: Pass 1 - UpdateHistories returned 0 symbols
08/05/2023 11:00:11: Pass 2 - Requesting updates for 1 symbols...
08/05/2023 11:00:11: Pass 2 - UpdateHistories returned 0 symbols
08/05/2023 11:00:21: Pass 3 - Requesting updates for 1 symbols...
08/05/2023 11:00:21: Pass 3 - UpdateHistories returned 0 symbols
08/05/2023 11:00:31: Pass 4 - Requesting updates for 1 symbols...
08/05/2023 11:00:31: Pass 4 - UpdateHistories returned 0 symbols
08/05/2023 11:00:41: Pass 5 - Requesting updates for 1 symbols...
08/05/2023 11:00:41: Pass 5 - UpdateHistories returned 0 symbols
08/05/2023 11:00:41: NextRun set to 08/05/2023 12:00
08/05/2023 11:00:41: Status = Incomplete
08/05/2023 11:00:41: No Updates for 1 Symbol: LYXE-USDT
08/05/2023 12:00:00: Status = Processing
08/05/2023 12:00:00: Pass 1 - Requesting updates for 1 symbols...
08/05/2023 12:00:00: Pass 1 - UpdateHistories returned 0 symbols
08/05/2023 12:00:10: Pass 2 - Requesting updates for 1 symbols...
08/05/2023 12:00:10: Pass 2 - UpdateHistories returned 0 symbols
08/05/2023 12:00:20: Pass 3 - Requesting updates for 1 symbols...
08/05/2023 12:00:20: Pass 3 - UpdateHistories returned 0 symbols
08/05/2023 12:00:30: Pass 4 - Requesting updates for 1 symbols...
08/05/2023 12:00:30: Pass 4 - UpdateHistories returned 0 symbols
08/05/2023 12:00:40: Pass 5 - Requesting updates for 1 symbols...
08/05/2023 12:00:40: Pass 5 - UpdateHistories returned 0 symbols
08/05/2023 12:00:40: NextRun set to 08/05/2023 13:00
08/05/2023 12:00:40: Status = Incomplete
08/05/2023 12:00:40: No Updates for 1 Symbol: LYXE-USDT
08/05/2023 13:00:00: Status = Processing
08/05/2023 13:00:00: Pass 1 - Requesting updates for 1 symbols...
08/05/2023 13:00:00: Pass 1 - UpdateHistories returned 0 symbols
08/05/2023 13:00:10: Pass 2 - Requesting updates for 1 symbols...
08/05/2023 13:00:10: Pass 2 - UpdateHistories returned 0 symbols
08/05/2023 13:00:20: Pass 3 - Requesting updates for 1 symbols...
08/05/2023 13:00:20: Pass 3 - UpdateHistories returned 0 symbols
08/05/2023 13:00:30: Pass 4 - Requesting updates for 1 symbols...
08/05/2023 13:00:30: Pass 4 - UpdateHistories returned 0 symbols
08/05/2023 13:00:40: Pass 5 - Requesting updates for 1 symbols...
08/05/2023 13:00:40: Pass 5 - UpdateHistories returned 0 symbols
08/05/2023 13:00:40: NextRun set to 08/05/2023 14:00
08/05/2023 13:00:40: Status = Incomplete
08/05/2023 13:00:40: No Updates for 1 Symbol: LYXE-USDT
08/05/2023 14:00:01: Status = Processing
08/05/2023 14:00:01: Pass 1 - Requesting updates for 1 symbols...
08/05/2023 14:00:01: Pass 1 - UpdateHistories returned 0 symbols
08/05/2023 14:00:11: Pass 2 - Requesting updates for 1 symbols...
08/05/2023 14:00:11: Pass 2 - UpdateHistories returned 0 symbols
08/05/2023 14:00:21: Pass 3 - Requesting updates for 1 symbols...
08/05/2023 14:00:21: Pass 3 - UpdateHistories returned 0 symbols
08/05/2023 14:00:31: Pass 4 - Requesting updates for 1 symbols...
08/05/2023 14:00:31: Pass 4 - UpdateHistories returned 0 symbols
08/05/2023 14:00:41: Pass 5 - Requesting updates for 1 symbols...
08/05/2023 14:00:41: Pass 5 - UpdateHistories returned 0 symbols
08/05/2023 14:00:41: NextRun set to 08/05/2023 15:00
08/05/2023 14:00:41: Status = Incomplete
08/05/2023 14:00:41: No Updates for 1 Symbol: LYXE-USDT
08/05/2023 15:00:01: Status = Processing
08/05/2023 15:00:01: Pass 1 - Requesting updates for 1 symbols...
08/05/2023 15:00:01: Pass 1 - UpdateHistories returned 0 symbols
08/05/2023 15:00:11: Pass 2 - Requesting updates for 1 symbols...
08/05/2023 15:00:11: Pass 2 - UpdateHistories returned 0 symbols
08/05/2023 15:00:21: Pass 3 - Requesting updates for 1 symbols...
08/05/2023 15:00:21: Pass 3 - UpdateHistories returned 0 symbols
08/05/2023 15:00:31: Pass 4 - Requesting updates for 1 symbols...
08/05/2023 15:00:31: Pass 4 - UpdateHistories returned 0 symbols
08/05/2023 15:00:41: Pass 5 - Requesting updates for 1 symbols...
08/05/2023 15:00:41: Pass 5 - UpdateHistories returned 0 symbols
08/05/2023 15:00:41: NextRun set to 08/05/2023 16:00
08/05/2023 15:00:41: Status = Incomplete
08/05/2023 15:00:41: No Updates for 1 Symbol: LYXE-USDT
08/05/2023 16:00:00: Status = Processing
08/05/2023 16:00:00: Pass 1 - Requesting updates for 1 symbols...
08/05/2023 16:00:00: Pass 1 - UpdateHistories returned 0 symbols
08/05/2023 16:00:10: Pass 2 - Requesting updates for 1 symbols...
08/05/2023 16:00:10: Pass 2 - UpdateHistories returned 0 symbols
08/05/2023 16:00:20: Pass 3 - Requesting updates for 1 symbols...
08/05/2023 16:00:20: Pass 3 - UpdateHistories returned 0 symbols
08/05/2023 16:00:30: Pass 4 - Requesting updates for 1 symbols...
08/05/2023 16:00:30: Pass 4 - UpdateHistories returned 0 symbols
08/05/2023 16:00:40: Pass 5 - Requesting updates for 1 symbols...
08/05/2023 16:00:40: Pass 5 - UpdateHistories returned 0 symbols
08/05/2023 16:00:40: NextRun set to 08/05/2023 17:00
08/05/2023 16:00:40: Status = Incomplete
08/05/2023 16:00:40: No Updates for 1 Symbol: LYXE-USDT
08/05/2023 16:08:00: Status = Deactivated
0
- ago
#5
since I couldn't add image under edit, here is the trenstrength level during strategy monitoring:

0
- ago
#6
Update: Streaming data seems to work fine now, not sure about Polling though.
0
Best Answer
Cone8
 ( 25.05% )
- ago
#7
The first thing Streaming does is poll to fill the history and obtain a partial bar. Polling has to work otherwise you'll be missing history when you start Streaming. Check the Log for the last DateTime in the history when activating the Strategy.
0
- ago
#8
thank you for information, won't touch the strategy now since it's running smoothly with streaming data, it did seem to happen on weekend, but that could just be a coincidence.
0
- ago
#9
So an update, the strategy worked fine, until 22:00 (+2 gmt) on friday. That was the last time limit order got automatically updated - after that, the problem with "incomplete" bars started happening, just like last weekend.



this also seems to correspond to stock market close.
0
Cone8
 ( 25.05% )
- ago
#10
My response - same as my first response in #1.

We can try to do better reconnecting after a streaming disconnect, but again, your best bet for 60-min bars is polling. And if that doesn't work, I still want to know what's in the Log Viewer.

In your Post #4, there was some valuable information in the Log Viewer. There was something about too many requests, something else about illegal characters, and something else about a null reference - but we don't know the source of the errors, because of the Data & time dialog on top of it.
1
Cone8
 ( 25.05% )
- ago
#11
Okay, looked into this in detail and I see what's happening with polling.

In general, Wealth-Lab doesn't request hourly bars. Instead, we request 30 minute bars and scale to hourly, mainly for the U.S. market that starts at 09:30.

The same happens for KuCoin, which actually does not support hourly bars anyway. So the hourly backfill using scaled 30-min bars works just fine.

The problem is when polling for the single hourly bar - WL is requesting an hourly bar, which, again, is not supported by KuCoin. We need to fix that to request the last two 30 minute bars and scale to hourly, like the backfill does.
1
- ago
#12
Thank you for looking into this, Cone :).

Yeah, I did have some other information in Log Viewer, but the illegal characters were for Binance, I did not seem to receive any errors at the times of incomplete fills. What is more interesting is that streaming data works all the time time, except right at the time of Stock market close - but after a few hours, it starts working again.

Not much happened in log viewer over last 2 days (I did not delete any messages)




this was the "null reference" error if it helps anything.

" at WealthLab.KuCoin.KuCoinHistorical.RequestHistory(String symbol, HistoryScale scale, DateTime startDate, DateTime endDate, Int32 maxBars, Boolean includePartialBar)
at WealthLab.KuCoin.KuCoinHistorical.GetHistoryInternal(String symbol, HistoryScale scale, DateTime startDate, DateTime endDate, Int32 maxBars)
at WealthLab.Data.DataProviderBase.GetHistory(String symbol, HistoryScale scale, DateTime startDate, DateTime endDate, Int32 maxBars, DataRequestOptions cb)
at WealthLab.Data.DataProviderBase.SelectSchema(String key)
at WealthLab.Data.DataProviderBase.GetQuote(String symbol)
at WealthLab.KuCoin.KuCoinBroker.UpdateAccounts()"
0
Cone8
 ( 25.05% )
- ago
#13
Thanks. We'll work on that one.

QUOTE:
What is more interesting is that streaming data works all the time time, except right at the time of Stock market close - but after a few hours, it starts working again.
Interesting indeed. I'll try to duplicate that too.

Something to check - make sure that Market: Crypto is selected at the bottom of the S. Monitor Item's Configuration.
0
Cone8
 ( 25.05% )
- ago
#14
Follow up -
KuCoin Build 6 should have improved reconnecting after a disconnect. What's your experience?
0

Reply

Bookmark

Sort