Yahoo! Data Not Updating for Last Session
Author: Cone
Creation Date: 3/21/2014 3:31 AM
profile picture

Cone

#1
Since this happens several times a year, I'll try to update this topic each time it occurs.

There are still 6 hours before the next session opens, but data for March 20 is still not available from Y! Finance. Get your backup provider ready. For another free provider, try Morningstar.
profile picture

kyokushin

#2
How Do I add Morningstar to the update list?
profile picture

Eugene

#3
Question is ambiguous:

* On how to install/uninstall/update Wealth-Lab extensions, review this illustrated tutorial and/or check out the Wealth-Lab User Guide > Extension Manager.

* On how to update data, see the User Guide: Data > Data Manager > Update Data.
profile picture

kyokushin

#4
Thanks, Eugene.

I installed Morningstar Extension but there are no bars added with EOD data.

Here's a fragment of the latest data update log of Morningstar symbols:

Updating Provider Morningstar Static ...
Provider update complete for Morningstar Static
----------------

Updating Provider Morningstar Fundamental Data ...
^AEX 83 fundamental items updated
...
ZMH 83 fundamental items updated
Provider update complete for Morningstar Fundamental Data
----------------

Update completed (1 min 34.6 sec)
----------------
profile picture

Eugene

#5
You've updated the fundamental data by checking "Morningstar Fundamental data" among "Fundamental data providers". The checkbox for static data is above, among "Historical data providers", and is called "Morningstar Static".
profile picture

kyokushin

#6
I updated Morningstar Static too. This was the outcome:

Updating Provider Morningstar Static ...
Provider update complete for Morningstar Static
----------------

Update completed (0.15 sec)




No bar added...
profile picture

Eugene

#7
For some reason, update all data may not work immediately for Morningstar data when you haven't used it before. Try highlighting your Morningstar DataSet and click "Update DataSet (Price)". This should kickstart it, and then data update will start working as expected.
profile picture

Cone

#8
It was a good run for the last 6 months, but Yahoo! is not updating again today for the 17 Sep 2014 session.
profile picture

cicerotullius

#9
any news here? I just tried it again its now after the closing of the session of the 18th of Sept and still there is no data for the 17th?

Might there be some next change in their API / Format as such things tend to occur some times? Ok it seems that on Yahoo finance page its not updated as well.
profile picture

Eugene

#10
For news you might want to contact Yahoo support. There are many backup data feeds if you need the data now.
profile picture

Cone

#11
As usual, it was just a temporary condition. Y! is updating to the prior session (18 Sept.) now.
profile picture

tallvira

#12
You can find some other sort of technical support services from yahoo as I got from ADVERTISING LINK REMOVED
One of the best free web mail service provider, well known to the world with the name of Yahoo! everyone is using yahoomail as with their communication medium

ACCOUNT BANNED FOR ADVERTISING
profile picture

Harapa

#13
This yahoo URL allows downloading last OHLCV data (as EOD, 15 min delayed) any time of the day.
http://quote.yahoo.com/d/quotes.csv?s=@@@&f=sd1ohgl1v&e=.csv
"@@@@" = Symbol/Ticker
Using this as a standby Yahoo provider will help to circumvent the "missing last EOD data". Off course one would need a provider "Add on" to use it in WL
profile picture

Cone

#14
In that case, open the Data Manager (Ctrl+M), go to the Yahoo! Data tab, and put a check next to "Always return data with partial bar". It works! Good show Harapa ;)
profile picture

Harapa

#15
Yes, indeed it works (except for Mutual funds...minor issue for most folks).
Glad I could help!
profile picture

Harapa

#16
Cone,
With the above URL one can download last data for symbols without history (for example ^VXX-IV). However, I get no data with Yahoo DL in WL (Partial bar checked). Wonder if this functionality can be Incorporated in future Yahoo provider. If there is a trick to get this even with the existing version, llet me know.
profile picture

Cone

#17
Yahoo! is updating.. .to 12/30/2015 without partial bar checked and to 12/31/2015 with it. Are you having trouble with a specific symbol? Please be more specific.
profile picture

Harapa

#18
Yes, there isn't any issue with yahoo updating except for symbols for which there is no history, like the one I mentioned.
I am interested in fetching data for this class (Intraday/EOD data yes, History No)..

Symbol examples
^VXX-IV
^XIV-IV
^NUGT-IV
SPXH16.CME
profile picture

Eugene

#19
There is nothing to fix or request here. All these symbols except the futures are happily streaming in a Quotes window. As per the FAQ, it is not possible to get the futures data from Yahoo. Neither is possible to obtain their historical data because Yahoo doesn't have it but it's not a problem for other providers.

Shouldn't Fidelity have symbols like .VXX in their data??
profile picture

Harapa

#20
QUOTE:
There is nothing to fix or request here.


My query highlighted the "possibility "of using the above URL to fetch these in WL. At the moment, I do this in another program but then I have to make those accessible in WL (lots of maneuvering...). So I asked...



QUOTE:
Shouldn't Fidelity have symbols like .VXX in their data??


No, checked already...

Actually, most are intraday NAV... examples tickers are for yahoo...other sources may use different ticker

profile picture

Eugene

#21
Re: lots of maneuvering

I don't see a good reason to include this URL in the Yahoo! provider as there's no history for those symbols anyway. If you need this for some purpose, you're better off creating a simple script for fetching the data on-the-fly. I already showed you how it can be done, so there's really no need for maneuvering or Excel:

Extracting Shares Related Info for Proshare ETF/Ns
WealthScript Techniques | Access Data from the Internet on-the-fly

/Should you have interest in discussing it please do so outside this thread that deals with "Yahoo! Data Not Updating for Last Session"./
profile picture

Eugene

#22
Many symbols are not up to date (missing July 13th prices). The Yahoo! Finance site seems to have had an overhaul recently. Maybe it's connected.
profile picture

Eugene

#23
Neopolix wrote in a duplicate thread:

QUOTE:
Hi,
i' ve got no Data today from Yahoo. Is that a General Problem?
profile picture

Eugene

#24
This is a problem at Yahoo's side. You might want to contact them (indicating that the "Download Data" link on the Historical Data page doesn't contain the data since Nov 7th) or use one of the many backup providers:

Report a data problem to Yahoo
profile picture

Neopolix

#25
Thanks Eugene!
profile picture

Eugene

#26
ATTENTION

Note that Yahoo does not seem to be returning the Dec 7 bar. Suggestion: use any backup data providers.
profile picture

Cone

#27
If you trade only Nasdaq 100 and S&P 100 stocks, give Wealth-Data a try!

Wealth-Lab Developer customers should already have the Wealth-Data extension installed with 6.9.15, so just sign up at the Wealth-Data site for access.
profile picture

Harapa

#28
Even since I activated "Always return data with partial bar"(Data Manger/Yahoo Data), Yahoo provider fetches data anytime of the day without any issue. Never a single missing day.
profile picture

Eugene

#29
Harapa, that's good advice. However, after not being able to update the data the traditional way today (last date is still 05/12/2017), I've used the partial bar technique and it couldn't update a few symbols:

QUOTE:
...
Security Names updated.
[1] AXP 11339 bars 15.05.2017 1 bars added
[3] CAT 13937 bars 12.05.2017 0 bars added
[6] DD 13937 bars 15.05.2017 1 bars added


So I had to update it again and this time it worked:
...
QUOTE:
Security Names updated.
[1] AXP 11339 bars 15.05.2017 0 bars added
[3] CAT 13938 bars 15.05.2017 1 bars added
[5] CVX 11951 bars 15.05.2017 0 bars added
profile picture

Harapa

#30
Today, in WL, I am not getting any data as well. This must be something to do with yahoo URL.
I am getting bad gateway error instead of no bars added
"CSCO Error: The remote server returned an error: (502) Bad Gateway"

as this works (on a different platform, though)
http://download.finance.yahoo.com/d/quotes.csv?s=@@@&f=sd1ohgl1v&e=.csv

but this worked this morning but stopped working EOD
http://chartapi.finance.yahoo.com/instrument/1.0/@@@/chartdata;type=quote;range=15d/csv
profile picture

Eugene

#31
QUOTE:
as this works (on a different platform, though)

This would work in Wealth-Lab too if we kept on using HTTP as in your link (we've switched to HTTPS). However, it's just the current day's snapshot quote - not very useful for historic backfills. Currently, I get a '504 Gateway Timeout Error' for all data update requests. Could be a glitch, could be depreciation of the old API. Let's keep an eye on this:

Yahoo Historical Data - Did they change the URL recently?
Is Yahoo! Finance API broken?
has yahoo stopped history data api service?

QUOTE:
but this worked this morning but stopped working EOD

We weren't using this RESTful API which reminds me of Google API.

Just noticed that the CSV download link on the Y! Finance website has changed to:
CODE:
Please log in to see this code.

The bad thing is this there's more under the hood than this vanilla URL. Now it requires cookie authorization and a "crumb" (the last param) which is sort of CSRF token.

Although there's a way to request the data without cookies it consumes too much network traffic and therefore may not be suitable for all.
profile picture

Neopolix

#32
Hi there,

what is your opinion. When does it work again?

profile picture

Eugene

#33
Hi,

It might work again some day when there's more clarity on what's happened. The outcome depends very much on what it is:

1) an outage? = wait for a few days until it starts working
2) old API decommissioned, API switchover? = fixing it may take a long time

For now it seems like the worst has happened.

At any rate, there's no shortage of alternative data providers. As here we need to focus strictly on the technical aspects of Yahoo APIs, you're welcome to discuss alternatives in other forum threads. Thanks.
profile picture

Harapa

#34
Yahoo Provider:
Working URL's for data download/retrieval as of 5/17/2017 AM. No cookies involved...
//@@@ or @@@%%% ...fills in the symbol

Last Trading only. Reports last price (intraday or EOD after market close))
http://download.finance.yahoo.com/d/quotes.csv?s=@@@&f=sd1ohgl1v&e=.csv

EOD...Fetches History
Defined Interval: (Need to define beginning and end dates...e.g 1/1/1950 to today). Note UNIX time stamp after"="
https://finance.yahoo.com/quote/@@@%%%/history?period1=-630961200&period2=1494993600&interval=1d&filter=history&frequency=1d

profile picture

Eugene

#35
Thanks for trying to help.

QUOTE:
EOD...Fetches History

I knew that but this is not suitable as every page is a 0.5MB request with tons of supporting stuff packed in whereas a correctly formed Y! request consumes 2 orders of magnitude less traffic (only a handful KB).

QUOTE:
Last Trading only. Reports last price (intraday or EOD after market close))

Question is for how long will it last.

On the bright side, the so called protection (cookie + crumb) is a joke compared to more sophisticated obstacles which we've overcame. The strikingly bad news is what I've just read on ET board:

1. Previously, OHLC were unadjusted and now OHL are split-adjusted
2. Sudden injections of random false data - I can confirm e.g. a couple of "null" bars in AAPL
3. Gaps of data (blocks or certain dates) - e.g. a hole of 143 bars in TRV

Time will tell whether this is intentional or just the effects of transition.
profile picture

Vadim_Kaplya

#36
Yahoo provider still does not have data (since 12th). It looks they could have changed the links for data fetching, please check it. WLD prints 504 error into the update log.Thank you
profile picture

Eugene

#37
Vadim, thank you for letting us know. Please scroll this page up to my post #29. In the conversation that followed you might find some new insight into the problem. Thanks again.
profile picture

Vadim_Kaplya

#38
Eugene, thank you. I apologise, It does not work, - I used to load a bunch of stocks every day from indices to check the candidates, now none of the universe is loading even partial bar.
profile picture

Eugene

#39
Chisholm's Third Law says, If you explain so clearly that nobody can misunderstand, somebody will. ;)

Vadim, the idea was that if you read the conversation that followed up to post #35 and not just #29, you'd know the verdict: old Yahoo data as we knew it is now broken for everyone in the world. The old API is discontinued. There can be no workaround. Now it's a major effort to adjust to the new API considering all its drawbacks known and unknown (see #35).
profile picture

Eugene

#40
An updated Yahoo provider is already being tested. We'll try to release it after we iron out some glitches.