Fidelity Static Provider doesn't update correctly
Author: kazuna
Creation Date: 6/23/2018 12:21 AM
profile picture

kazuna

#1
Fidelity Static Provider seems not updating correctly.
Has anyone else noticed it?

I wipe FidelityStaticProvider and FidelityFMDFundamentalProvider then perform data update.


[Case 1]
Some symbols are not update at all with an output saying like:
Updating 1 Minute data ...
TNA 0 bars No data 0 bars added in 2 chunks

Performing data update little later works in most cases:
Updating 1 Minute data ...
TNA 189834 bars 2018/06/21 16:00 189834 bars added in 13 chunks


[Case 2]
Some symbols are updated but partially until 4/20:
Updating 1 Minute data ...
UDOW 188015 bars 2018/04/20 16:00 188015 bars added in 19 chunks

Performing data update little later gives an update:
Updating 1 Minute data ...
UDOW 203475 bars 2018/06/21 16:00 15460 bars added in 2 chunks


[Case 3]
Some symbols are updated but partially after 4/20:
Updating 1 Minute data ...
FXI 16763 bars 2018/06/21 16:00 16763 bars added in 2 chunks

Performing data update doesn't give an update because it doesn't update the bars in the past.


I have seen this behavior in the last couple of days at least, maybe more.
It doesn't matter whether it is automatically updated at 4am or manually updated at 6pm.

There apparently something going wrong with Fidelity Static Provider update.
Be aware!!!
profile picture

Eugene

#2
Thanks for letting the forum know. Fidelity customers are welcome to report this problem to Fidelity so that they're aware!

Below are some recent reports of the same kind. One of them indicates corrupt data being returned by a server:

Fidelity data 0 bars when updating SPY 1 minute DataSet
Fidelity 1 minute static data stopped updating
profile picture

kazuna

#3
This issue is even worse than anything else like those recent reports and the split stock issue because wiping cached data won't help.
Also it's hard to workaround this issue because performing data update multiple times won't help as explained in [Case 3] above.
profile picture

Eugene

#4
Yes, I realize this. This is why it's so important to give Fidelity a heads-up. Your account may be hitting a server with corrupt data.
profile picture

kazuna

#5
It doesn't look like corrupt data because there definitely some patterns I can see:

[Case 1] no bar data
[Case 2] bar data ends 2 months ago
[Case 3] bar data starts 2 months ago

Roughly 90% symbols are updated correctly but other 10% symbols are randomly chosen to cases above.
I am seeing this behavior on not only single machine but all of my machines these days.

Anyone seeing it?
profile picture

kazuna

#6
The examples above are 1 Minute data but this is happening with Daily data as well.

[Case 1]
Some symbols are not update at all with an output saying like:
Updating Daily data ...
UPRO 0 bars No data 0 bars added

Performing data update little later works in most cases:
Updating Daily data ...
UPRO 2265 bars 2018/06/22 2265 bars added

So far I see only [Case 1] with Daily data.
[Case 2] and [Case 3] seem to be only with 1 Minute data.
profile picture

kazuna

#7
Here is another pattern to [Case 1].

Updating 1 Minute data ...
USLV 1682 bars 2018/06/22 1682 bars added in 42 chunks

Updating 1 Minute data ...
USLV 427279 bars 2018/06/22 16:00 427279 bars added in 42 chunks

First update was missing the time (16:00) and Symbol Details showed 0 bar for USLV despite the update saying 1682 bars.
Immediately after that, second update went fine.
profile picture

Eugene

#8
Just thinking out loud:

QUOTE:
Anyone seeing it?

Since nobody else seems to be seeing it, I can think of two culprits: your account is hitting a server 'gone bad' in the Fidelity data farm or something about your environment. Faulty drive or 3rd party software could be an explanation but since it happens on multiple PCs this may be related to your workflow instead. What's specific to them all?

Or better, could you set up a virtual machine with nothing but WLP inside and run updates there? You can get a VM with preinstalled Windows for your preferred VM player free from Microsoft at https://developer.microsoft.com/en-us/microsoft-edge/tools/vms/.

QUOTE:
[Case 1]
Some symbols are not update at all with an output saying like:
Updating Daily data ...
UPRO 0 bars No data 0 bars added

UPRO recently had a split (this May) but it wasn't a reverse one (which we believe is connected to the infamous "Fidelity data can not be updated except for X minute" bug.
profile picture

kazuna

#9
QUOTE:
What's specific to them all?
Nothing specific. Every night I wipe all Fidelity Static Provider folders and let automatic update to download all symbols from scratch. That's probably why I noticed this new issue.

QUOTE:
could you set up a virtual machine with nothing but WLP inside and run updates there?
I have 7 machines running WLP. 2 of them are running under VMware for testing WLP update before deploying on other machines. All 7 machines are seeing the same issues. 4 machines are very clean and they have nothing running other than WLP.

QUOTE:
UPRO recently had a split (this May) but it wasn't a reverse one
I was one of the first customers who ran into that issue (update fail after split). For that reason, I deployed the workaround which is to wipe all Fidelity Static Provider folders and let update to download all symbols from scratch. The split stock update issue has gone away completely. Unfortunately, wiping the folders doesn't workaround this new issue.

I bet more people will notice the issue once they happen to download symbols from the scratch.
profile picture

Eugene

#10
Sorry that you're experiencing the issue. Perhaps it's the download from scratch that makes your case specific? What if you try to wipe just the folder of the affected timeframe instead? (I realize that you're after more automated solutions but the data update log could be parsed by e.g. an AutoIt script to attempt find data remaining not up to date after an update, affected timeframe's data folder purged, and download repeated. Just for example).

P.S.
QUOTE:
UPRO 0 bars No data 0 bars added

We already have reports proving that this UPRO case is data corruption:

https://www.wealth-lab.com/Forum/Posts/Fidelity-data-0-bars-when-updating-SPY-1-minute-DataSet-39319
https://www.wealth-lab.com/Forum/Posts/Fidelity-1-minute-static-data-stopped-updating-39346 - the corrupt file contains Fidelity fundamental data

When it happens you might want to peek inside UPRO.WL to find remnants of unsuitable fundamental data there. I can't claim for sure that this is a server side error (i.e. Fidelity server returns mixed static/fundamental data causing .WL file corruption) but I have a hard time imagining how would the Fidelity fundamental provider affect the static data. IMHO it's just not programmed for that. But server occasionally returning flawed data to WLP would explain it. I can think of a troubleshooting procedure to analyze the data being transmitted - you might want to create a support ticket if interested.
profile picture

kazuna

#11
QUOTE:
What if you try to wipe just the folder of the affected timeframe instead?
I tried something like that but it doesn't work well because each time some symbols are randomly not updated correctly.

QUOTE:
We already have reports proving that this UPRO case is data corruption:
UPRO was just an example. As you can see the LastUpdateLog.txt attached, I ended up repeating update 4 times until I see no symbols are updated any more.

Updating Daily data ...
IWM 0 bars No data 0 bars added
UPW 0 bars No data 0 bars added
URE 0 bars No data 0 bars added
SCO 0 bars No data 0 bars added
SDP 0 bars No data 0 bars added
SDS 0 bars No data 0 bars added
SIJ 0 bars No data 0 bars added
SKF 0 bars No data 0 bars added
SMN 0 bars No data 0 bars added
SRS 0 bars No data 0 bars added
SSG 0 bars No data 0 bars added
TBT 0 bars No data 0 bars added
TWM 0 bars No data 0 bars added
BRZU 0 bars No data 0 bars added
DRN 0 bars No data 0 bars added
FAS 0 bars No data 0 bars added
GASL 0 bars No data 0 bars added
SPXL 0 bars No data 0 bars added
TECL 0 bars No data 0 bars added
TMF 0 bars No data 0 bars added
TNA 0 bars No data 0 bars added
TQQQ 0 bars No data 0 bars added

Updating 1 Minute data ...
GDX 0 bars No data 0 bars added in 2 chunks
QQQ 0 bars No data 0 bars added in 2 chunks
SOXX 0 bars No data 0 bars added in 2 chunks
SPY 0 bars No data 0 bars added in 2 chunks
NUGT 0 bars No data 0 bars added in 2 chunks
SPXL 0 bars No data 0 bars added in 2 chunks
TMF 0 bars No data 0 bars added in 2 chunks
YANG 0 bars No data 0 bars added in 2 chunks

QUOTE:
and download repeated.
Even repeating the update doesn't fully solve the problem because there are symbols updated with much fewer bars. These symbols are supposed to have 40 to 80 chunks but only getting several chunks.

Updating 1 Minute data ...
DIA 79793 bars 2018/06/22 16:00 79793 bars added in 6 chunks
EEM 80158 bars 2018/06/22 16:00 80158 bars added in 6 chunks
FXI 95189 bars 2018/06/22 16:00 95189 bars added in 7 chunks
GDXJ 79703 bars 2018/06/22 16:00 79703 bars added in 6 chunks
IBB 62263 bars 2018/06/22 16:00 62263 bars added in 5 chunks
IWM 15990 bars 2018/06/22 16:00 15990 bars added in 2 chunks

QUOTE:
you might want to create a support ticket if interested.
Yes, I am interested.
profile picture

Eugene

#12
Thanks for your detailed problem reports. Please follow instructions in your support ticket.
profile picture

kazuna

#13
I just confirmed that wiping the data folders and updating from scratch doesn't matter for the problem.
I didn't wipe the data folder this morning and I performed the update but some symbols are not getting any bars (0 bars added) since the last update.

Updating 1 Minute data ...
EDZ 69698 bars 2018/06/22 16:00 0 bars added
TMF 30138 bars 2018/06/25 16:00 257 bars added
TMV 264392 bars 2018/06/25 16:00 319 bars added
TNA 383300 bars 2018/06/22 16:00 0 bars added
TQQQ 64225 bars 2018/06/25 16:00 390 bars added
TZA 95933 bars 2018/06/25 16:00 390 bars added
UDOW 31272 bars 2018/06/25 16:00 385 bars added
UGAZ 30145 bars 2018/06/25 16:00 350 bars added
UGLD 232370 bars 2018/06/22 16:00 0 bars added
USLV 427279 bars 2018/06/22 16:00 0 bars added
UVXY 33149 bars 2018/06/25 16:00 390 bars added
profile picture

kazuna

#14
The problem has gone away completely for the all updates today!
Apparently Fidelity fixed the problem.
profile picture

kazuna

#15
All symbols are updated correctly today but there is one error found in LastUpdateLog.txt.
QUOTE:
YANG Error: Unable to connect to the remote server


Performing update one more time corrected the problem.
QUOTE:
YANG 236784 bars 2018/06/28 16:00 236784 bars added in 44 chunks

profile picture

Eugene

#16
Could be a temporary network issue on your end.
profile picture

kazuna

#17
Everyone be aware that this problem has NOT gone away completely yet!

The problem has gone away from the intra-day symbols so far.

However, the problem came back to daily symbols today.
I had to perform the symbol update twice at least.

[PC #1]

Updating Daily data ...
QQQ 0 bars No data 0 bars added
XLV 0 bars No data 0 bars added
XLY 0 bars No data 0 bars added
YXI 0 bars No data 0 bars added
DDM 0 bars No data 0 bars added
EFO 0 bars No data 0 bars added
UCO 0 bars No data 0 bars added
UGL 0 bars No data 0 bars added
DZZ 0 bars No data 0 bars added

Updating Daily data ...
QQQ 4865 bars 2018/07/09 4865 bars added
XLV 4917 bars 2018/07/09 4917 bars added
XLY 4917 bars 2018/07/09 4917 bars added
YXI 1994 bars 2018/07/09 1994 bars added
DDM 3033 bars 2018/07/09 3033 bars added
EFO 1918 bars 2018/07/06 1918 bars added
UCO 2420 bars 2018/07/09 2420 bars added
UGL 2415 bars 2018/07/09 2415 bars added
DZZ 2606 bars 2018/07/09 2606 bars added

[PC #2]

Updating Daily data ...
SPY 0 bars No data 0 bars added
IEFA 0 bars No data 0 bars added
JNK 0 bars No data 0 bars added
REK 0 bars No data 0 bars added
REW 0 bars No data 0 bars added
JDST 0 bars No data 0 bars added

Updating Daily data ...
SPY 6407 bars 2018/07/09 6407 bars added
IEFA 1434 bars 2018/07/09 1434 bars added
JNK 2666 bars 2018/07/09 2666 bars added
REK 2083 bars 2018/07/09 2083 bars added
REW 2776 bars 2018/07/09 2776 bars added
JDST 1199 bars 2018/07/09 1199 bars added
profile picture

kazuna

#18
Daily symbols update went fine this morning.
I will keep my eyes on it.
profile picture

psimmons

#19
I have also been having issues with "sluggishness" of intraday data over the last few days, in my case 30min. It appears to me to be on the Fidelity side. It appears to be for certain symbols, a significant number but not all. Re-running the strategy window overcomes the issue suggesting (atleast for me) that it is slow data, rather than corrupt data.

I am not having execution issues on my 1min streaming data, so this is currently an irritant for me.
This website uses cookies to improve your experience. We'll assume you're ok with that, but you can opt-out if you wish (Read more).