Difference in Execution Times - Streaming Chart Window vs. Strategy Monitor
Author: chamoun
Creation Date: 12/7/2011 12:59 PM
profile picture

chamoun

#1
I am running a real-time strategy (1m data) that takes about 4.5 seconds to update in a Chart Window in streaming mode. When I run the same strategy in the Strategy Monitor it takes 20 seconds to update. This is a huge time difference when every second matters. This difference occurs when nothing is running on the machine and regardless of whether I run the Chart and Strategy Monitor concurrently or alone. I am puzzled because I thought the Strategy Monitor was supposed to deliver the most streamlined execution path. I need to run the strategy on multiple accounts so multiple charts are not an option.

Please advise.

Thanks.

System (Win7 x64, I7 2600K 3.4GHz, 16GB)
profile picture

Eugene

#2
Delays in Strategy Monitor are expected. To quote the Wiki FAQ:

QUOTE:
If you're a user of Fidelity Wealth-Lab Pro 6.2 trading less than 12-15 symbols, we suggest using Streaming strategies. Streaming Strategy windows execute immediately at the end of the bar w/o delay, unlike Strategy Monitor (see Help - User Guide > Strategy Monitor > Employment Notes).

For more instruments, consider using the Strategy Monitor. Wealth-Lab Pro users should know that it makes requests for Fidelity data in sets of 10-symbols, so to minimize data request delays, break up your DataSets in groups of 10 symbols and add a Strategy Monitor item for each one. The Strategy Monitor is recommended for pair trading Strategies.

Maybe in 6.3 it changes to the better with introduction of Fidelity Streaming quotes in Strategy Monitor, but in 6.2 it's normal because there's inherent delay by design.
profile picture

chamoun

#3
Thank you for the clarification. Streaming quotes will be a huge upgrade to WL.

It is a very powerful tool but the data streams are way behind the capabilities of the system and represent a major impediment to real-time trading strategies.

Eagerly awaiting the 6.3 release!

Thanks again for providing to notch and very responsive support.
profile picture

Cone

#4
Just a clarification for posterity. Intraday data responsiveness in the Strategy Monitor should be greatly improved in 6.3, but there will still be some delay over using a streaming chart. Hopefully the delay is less than 10 seconds, but I expect at least 3 seconds. Even with the delay, however, all the symbols should be updated practically simultaneously. The User Guide will have the details.

If you're trading just a handful of symbols, streaming will always be the fastest way to get an order to market.
profile picture

chamoun

#5
Understood - The challenge I have is that I use Auto Trading and once I am done testing it will involve Multiple Symbols and Multiple Fidelity Accounts.

Unless I am missing something it does not appear that you can direct the Trades to a particular account for each chart the way you can from the strategy monitor (if there is a way please let me know). For my testing phase running in chart mode is fine but sometimes the chart does not update and I worry that it may drop a trade - the strategy monitor seems more robust. A reduction in delay will be welcome and I hope at some point in the future Fidelity will consider providing support for a major data provider like Esignal.

Fidelity's handling of realtime data is awful - you can not get a realtime profit & loss in Active Trader Pro without hitting an update button. In fact I am not aware of any Fidelity platform that will allow you to track in realtime profit & loss. In WealthLab the cost basis disappears when I hit the update button in the Account Balances window. All their systems appear to have cost basis amnesia.


Thanks.
profile picture

Eugene

#6
QUOTE:
Unless I am missing something it does not appear that you can direct the Trades to a particular account for each chart the way you can from the strategy monitor (if there is a way please let me know).

In the "Open Strategy" dialog, highlight a Strategy and choose "Set account..."
QUOTE:
I hope at some point in the future Fidelity will consider providing support for a major data provider like Esignal.

Why Fidelity, being a data vendor themselves, would consider a 3rd party data provider? (Rhetorical question). Nonetheless, our team had started building an Esignal data provider but due to technical issues/incompatibilities and prioritizing another (highly important) project the development of the provider has slackened. Next year we may have the time to look into it.
profile picture

chamoun

#7
Excellent - set accounts works. One issue though is that setting strategy parameters other than the default and saving a chart workspace does not retain those setting when the workspace is reopened. Is that the case or am I doing something wrong?


It appears that separate Workspaces run in separate threads & Different strategies in the strategy monitor also run in separate threads - Correct? Just trying to optimize execution time.


Regarding the data provider issue Fidelity seems to miss the point that in today's markets time is MONEY and their data streams are sluggish and way behind the competition - they should beat them or join them their goal should be to provide their customers the fastest and most reliable EXECUTION & REPORTING. Probably preaching to the converted here.

Thanks.
profile picture

Cone

#8
QUOTE:
setting strategy parameters
Make sure to "Save Parameters" after you set them to change the default. (Note: the default in the script code is not affected by saving parameters, just the default that is applied.) If you use different parameters for different symbols, then you need to use Preferred Values.

Re: threading, correct

Fidelity streaming is second to none. I've used them all, and it's the best I've seen since the 6.0 upgrade (although it hasn't been without a few bugs that were worked out along the way). Try monitoring the Russell 3000 in a Quotes window in Wealth-Lab, no problem.

On the other hand, Fidelity is addressing the Strategy Monitor issue that uses polling to request static bars. When there are more than 10 stocks in the same S. Monitor item, then there will be excessive delay due to their static provider design. In 6.3 that will be corrected/enhanced in a manner that no other provider that I know of can match... how good it is will depend on the back end service, which will have its challenges to overcome, but they should succeed.
profile picture

chamoun

#9
Thanks for the Tip on Save Parameters - I never used it because I thought it updated the code.

Regarding the streaming data I will take your word for it in terms of the quality and speed of the source. I have never had an issue with the Quotes Window however it is of limited use to me because I am completely reliant on Auto-Trading and hardly ever enter an order manually unless something goes wrong and I need to correct a position.

The issue then comes down to implementation - The streaming mode in the Chart window works great but the BIG problem there is that the strategy stops executing the current minute if there is a conflict with incoming data and if at that minute you would have entered or exited a position then you miss it and this a problem for Auto-Trading based strategies. So while I have switched to the Chart window for my strategy I am extremely uneasy and currently spend more time than necessary monitoring the Chart window to make sure nothing gets missed.

The Strategy Monitor is a lot more reliable when it comes to making sure that all minutes execute and therefore nothing is missed - the problem is the delay and going down to ~3 seconds from 10 is a big plus but I am still puzzled at why there is any delay if you are capturing the stream as it is coming and deliver the most up to date values to the strategy when the minute rolls over the way it is done in the Chart window.

It seems to me that if WL had a streaming quote engine running in the background in a separate thread a lot of the issues can be avoided in the Strategy and Streaming Chart windows. Using ESignal in effect provides an isolated data stream that is ON all the time and that is not subject to issues that can arise from Strategy handling & execution.

In today's market every second matters - I am sure Fidelity would not be comfortable providing their traders with a data source that carries a few second delay.

Thanks again for your help.
profile picture

Cone

#10
QUOTE:
if there is a conflict with incoming data
There's always a risk that something can go wrong anywhere from the back-end systems to the internet backbone to your computer shutting down spontaneously (which actually happened to me last week after a Dell tech replaced a motherboard). But "conflict"? In 6.2 build 46, there are no known problems with streaming (other than the fact that secondary symbols in a strategy are not streamed). If you're using 6.2.46 and have trouble with streaming charts, please create a support ticket.

QUOTE:
I am still puzzled at why there is any delay if you are capturing the stream
We're talking about the 6.3 S. Monitor here: Because it's not streaming. Streaming for hundreds of symbols is an enormous amount of overhead to just get 5 values at the end of the bar. All that overhead is going to be done once on the back end and the S. Monitor will just get the result without polling for it, but there will be back-end processing and transmission delays.

QUOTE:
In today's market every second matters
I disagree. You can't complete at the "second" level. Sometimes those seconds will work against you, but I know they work for you too. It would be an interesting case study to show the price distribution 5, 10, and 15 seconds after the closing interval mark. Nonetheless, I would tend to agree that 10 or more seconds of delay is excessive for trading 1-minute bars.

Anyway, if you feel that your strategy needs every second, then use streaming charts. Case closed. If streaming charts are not reliable for you with 6.2.46, then we want to know about it.
profile picture

chamoun

#11
QUOTE:
Aborted Thread/ conflict with incoming data

This issue occurred on & off in prior releases which lead me to move to the S. Monitor because I assumed it was the nature of the beast - based on this exchange I have moved back to the Streaming Chart window and I have not encountered the issue in the last couple of days. I will report the problem if it occurs in 6.2.46. Will secondary Symbols Stream in 6.3?

QUOTE:
S. Monitor Data

My bad - I assumed that 6.3 will use a streaming server but now I understand better the implementation. This is a major step forward but I still believe that a product like Wealth-Lab should have the capability of a local streaming server. Such a server should be provided by Fidelity or through access to a leading Third party platform like ESignal. Users who need this type of data not only will be willing to pay for it but also can provision their computing and internet resources to accommodate it.

QUOTE:
Seconds Matter

I agree with your position when it comes to an equally distributed random sample of minutes across stable, rising and declining markets. However if your strategy's accuracy at detecting an inflection point is >50% than on average the longer you wait the more slippage you encounter. The overall $ impact will depend on the overall accuracy, the size of the trade and how fast the market is moving. In my case accuracies are > 60%, my trades are large and the big variable I do not control is the rate of change in the market. In low volatility markets it does not matter but when things are moving it can be costly.

Thank you for responding on a Sunday - You and Eugene are a role model for knowledgable and dedicated support.
profile picture

Cone

#12
QUOTE:
Will secondary Symbols Stream in 6.3?
No. The fundamental problem here is that the script has to execute to determine if there's a secondary symbol. So the design was always to create a static request for secondary data during execution. But we later found that this request most often occurs before the most-recent static bar is built on the back-end, and therefore you must artificially add a delay to the strategy, i.e., Sleep 3 or 4 seconds, so that you get the most-recent secondary bar. That said, it's still seen as a deficiency and is on the radar to fix, but it will require some architectural changes.

QUOTE:
I assumed that 6.3 will use a streaming server
The confusion is because somewhere it may have been mentioned that the new data will come from the streaming provider, which is correct. But the implementation in that provider's black box will not use streaming, per se, and will be as I previously described.

QUOTE:
if your strategy's accuracy at detecting an inflection point is >50% than on average the longer you wait the more slippage you encounter.
Understood. Good point.
This website uses cookies to improve your experience. We'll assume you're ok with that, but you can opt-out if you wish (Read more).