Morningstar fundamental, static and streaming data provider
Author: abegy
Creation Date: 4/27/2012 4:33 AM
profile picture

abegy

#1
I see that in the last version of the MS123 Extra Fundamental/News providers extension, you have added the possibility to download dividend from Morningstar.

Is it possible for the next release to add more fundamentals data from this source ?

Morningstar is a worldwide database. Moreover, you can download fundamentals data directly on a csv file (see export feature on the following link)

http://financials.morningstar.com/ratios/r.html?t=BNP&region=FRA&culture=en-US
http://financials.morningstar.com/income-statement/is.html?t=BNP&region=FRA&culture=en-us

If you have the possibility to add also Morningstar has an historical/streaming Datasource extension, it would be nice for an international trader because :

- Quality of data is very good
- There is more than 10 years online of historical data
- 5 years of fundamentals data (more than 20 differents items
- There is realtime streaming for some markets
- it's free of charge

For an international trader like me, there is no others source at this time in WLD where I can match streaming/Historical/Fundamentals data because international symbol code are all the time differents depending of the source as I have explained several time here.

I know that I can develop my own dataprovider but what I would like is to spend my time is on trading strategy development and not on technical modules

WLD is a nice tool with a great support
Give for international traders a minimum of facility to trade on their local markets ;-)

Thanks for you understanding and I hope you will be able to do something :-)
profile picture

Eugene

#2
Good day,

Some background on why did I pick Morningstar's dividend feed. Mainly for four reasons:

1. relative simplicity of implementation (compared to other Morningstar data items).
2. its exclusive dividend data fields missing in Fidelity's data.
3. the historical data depth surpasses Fidelity's (in this particular case).
4. based on experience, it might take a lot of time to introduce a change to Fidelity data feed.

Unfortunately, the data items you're proposing is a different story. As you pointed out, symbol names are different across different data providers. Wealth-Lab wasn't designed with symbol aliases in mind, and this feature is not on our radar this year. I think you'd agree with me that this Morningstar fundamental data provider would hardly be useful for EU stocks without a companion static data provider. Not to mention that the same stock data can be obtained from Yahoo! (BNP.PA) with full precision: for unknown reason, Morningstar rounds volume in their data e.g. "32.07 Mil".

Besides duplicating our effort of development, these Key Ratios are a challenge to parse. At first glance, it seems that the exported data presentation may require a sizable change of the design pattern I've been successfully using for other fundamental providers. Lastly, it's annual data like the already existing MSN/Yahoo fundamental providers.

I haven't made a decision yet, but chances of implementing this feed look pretty remote as of today.
profile picture

Eugene

#3
If we focus only on getting a subset of fixed items from "Key Ratios", then I think it's doable (and even implementing the companion static provider). Please create a support ticket and refer to this thread so it's not forgotten.
profile picture

Eugene

#4
QUOTE:
- Quality of data is very good

I. Quality of their static data is rather poor. How about lots of fake bars like this one particular example?


Now compare this with Yahoo, no sign of the fake 1/16/2012 bar:



Notice the different open price. This ain't some pink sheets, that's AAPL Nasdaq.

Not to mention that their static data (like Google) does not contain the open price in the 1990s.

And the worst thing, you could not weed out the fake bars if their volume is "0.0 Mil" - just because lots of perfectly valid symbols do have this "fake zero volume" too even though they were trading. But since the folks at Morningstar decided to lose precision by virtue of rounding, a stock still has a "0.0 Mil" volume if say 50,000 shares changed hands.

II. Their fundamental data suffers from two connected issues:

* Loss of true announcement date. For instance, fiscal year ended at 23 September but the data are marked as September - w/o the date.
* No date by default means 1st of month which is dead wrong if the data became available around the end of month. Consequently, arising possibility of peeking into the future. This is a serious problem that will be handled exactly like we're used to do with YCharts.
profile picture

Eugene

#5
Their ASCII Export produces the most malformed CSV data I've ever seen in years:

* Systematic violation of CSV standard RFC 4180 e.g. guideline #2.4
* Bad formation of interim headers (sometimes split in two lines, somewhere not)
* Multiple currency separators in the file...
* ...with subsequent need to wrap some values in double quotes
* No uniform section headers
* No uniformity in dates in the headers. Could start/end as "TTM" or "Latest Qtr"
* Essentially duplicate fields: "Revenue"
* As the result, an irregular structure preventing from creating a robust parser. It could be done by making several assumptions about the particular sections, so end result can break at any time if data vendor considers a change.

Example
profile picture

Eugene

#6
Nevertheless, I'll build the static and fundamental providers.
profile picture

abegy

#7
Thanks a lot ! Are you going to build also the streaming ?
profile picture

Eugene

#8
No, streaming is not considered.
profile picture

abegy

#9
Why ? As you know, the problem with international stocks is the code translation.
If you don't consider it, we will have no possibility to scan in realtime orders.
It's petty because you have it with BBFree, yahoo and google extension. So, why not with Morningstar ?
Do not forget that Morningstar provides some realtime quotes depending of the market.
profile picture

Eugene

#10
Well, maybe I was looking in the wrong place but there doesn't seem to be any sign of free streaming data. Can you point me?
profile picture

abegy

#11
In the quote page, there is the following note :
QUOTE:
ll data supplied by Morningstar Real-Time Data. U.S. intraday real-time exchange quotes are sourced from BATS when available. End-of-day quotes for Nasdaq, NYSE, and Amex securities will appear 15 minutes after close. Graph times are Eastern Standard.
profile picture

Eugene

#12
If you want me to take a look, you have to point me to the place where it's streaming, and what is required to get there.
profile picture

abegy

#13
In the quote page, http://quote.morningstar.com/stock/s.aspx?t=XPAR%3aBNP, you can see that the last price is updated automatically every second.
profile picture

Eugene

#14
Yep, I see their streaming quotes now that European market is open. Wasn't streaming for me before because I must had been visiting the page outside of EU market hours. Thanks.

Wish my JavaScript reverse engineering skills were better as the ins and outs of quote processing seems to be obfuscated. Nonetheless, it's clear to me how it's made. Let's see if I can work it out...
profile picture

Eugene

#15
Here's good news for you abegy: Morningstar Streaming provider will also be built.
profile picture

abegy

#16
Thanks a lot ! :-)
profile picture

Eugene

#17
The Morningstar Static (EOD), Streaming and Fundamental data provider for U.S. and international securities has been released. Registered Wealth-Lab customers please download and install it from Extensions:

Morningstar data provider

This thread is the place to discuss the provider. Make sure you've read the online guide before asking questions or reporting errors:

Morningstar Static and Streaming provider
Morningstar Fundamental Provider
profile picture

abegy

#18
I try to download fundamentals data for XPAR:BNP. When I update data, I have no problem (see the log here).

CODE:
Please log in to see this code.


But when I try to put a Fundamental item on a chart (for example dividend), I see nothing in the specific chart pane.
Is anything wrong ?
Thanks for your help.
profile picture

Eugene

#19
Reason: the fundamental items from "Morningstar Fundamental Data" haven't downloaded. Server did not return any valid data. (The number of fundamental data items updated is not indicative in this case. By design.)

With Morningstar being a free data vendor, this can and will occasionally happen. I faced it a couple of times during the development. But don't give up after first try. Retry and the server will eventually return valid data.
profile picture

Eugene

#20
The fundamental data feed has slightly changed their output format, rendering a dozen of key fundamental items blank (e.g. "Earnings Per Share"). A fix is in the works.
profile picture

abegy

#21
Do you have an example of the change ? I ask you because I haven't seen any change
profile picture

Eugene

#22
They started attaching currency (EUR) and/or volume qualifier (e.g. Mil) to exported CSV files. For example:

Was: Revenue
Is: Revenue USD Mil

Consequently, the provider is no longer able to match the string against its list of fundamental items.
profile picture

Eugene

#23
The Morningstar provider has been upgraded to 2012.11.1

* Fix: (Fundamental) almost dozen of key Financial items stopped working (zero values) due to a format change at Morningstar
* Change: (Fundamental) "Shares Mil" item was renamed "Shares"
profile picture

abegy

#24
Eugene, I see in the fundamental items that there is 2 items for morningstar Dividend :
- one with the morningstar logo
- another one with "f" as the logo

The first item report differents date for dividend operations but 0 for the cash dividend
The second one report only one date (I don't know for which operation) and the correct cash dividend amount.

For example, try with XPAR:BNP to see that.
Is it normal ?
profile picture

Eugene

#25
Yes, it is normal. Reporting 0 is not normal though (investigating). Will post in the proper forum thread.

There are two independent Morningstar fundamental providers, and each one has a dividend item:

* Morningstar extended dividend data ("[ms] dividend") - part of MS123 Extra Fundamental/News Providers
* Morningstar static/streaming/fundamental provider ("[ms] Dividends", aka "the one with f") - this provider

p.s. The "f" is is Wealth-Lab's default fundamental item icon that appears because the obfuscator has swallowed the correct icon.
profile picture

Eugene

#26
The Morningstar provider has been upgraded to 2012.11.2

Another hotfix:

* Change: (Fundamental) chart popups of fundamental items will display their title (to be easily recognized on the chart among 82 items)
* Fix: (Fundamental) fundamental icons appear as "f" (missing glyph)
profile picture

abegy

#27
Hi Eugene,

Is it possible to add the "TTM" values for all fundamentals items as the last value of the current fundamentals items ?
For example, I would like to see for XPAR:AC, -2.40 for Earnings per share instead of 0.12 in the current version of the extension.

Thanks a lot.
profile picture

Eugene

#28
Alexandre, TTM values will be displayed in upcoming release of the provider (no ETA right now).
profile picture

Eugene

#29
The Morningstar provider has been upgraded to 2013.05

Added TTM values (trailing 12-month).
profile picture

abegy

#30
Hi Eugene,

When I try to update historical quotes from Morningstar extension to have the last up to date data, I see that not all symbols are updated the first time. I need to update 2/3 times more to have it all.

Is it normal ?
profile picture

Eugene

#31
I have no idea. Many factors may play a role, like network connectivity, or free data feed limitations (did you know that some of them don't like parallel or sequential downloads so much that in the past one could even get 'banned' at Yahoo?). It's a free data feed, and when you update thousands of symbols, anything can happen.
profile picture

abegy

#32
I try to update only 40 symbols. See the log. As you will see

QUOTE:
Daily data is being updated...

Updating Provider Morningstar Static ...

Up-to-date symbols: 0, Update required for: 42, New symbols: 0
Symbol: XPAR:ALU, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:BNP, Bars: 0
Symbol: XPAR:RNO, Bars: 0
Symbol: XPAR:SPI, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:UFF, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:AC, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:ACA, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:AI, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:ALO, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:BN, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:CA, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:CAP, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:CS, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:DG, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:EAD, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:EDF, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:EI, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:EN, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:FP, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:FTE, Bars: 0
Symbol: XPAR:GLE, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:GSZ, Bars: 1, Last date: 7/15/2013
Symbol: XPAR:LG, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:LR, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:MC, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:ML, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:MT, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:OR, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:PP, Bars: 0
Symbol: XPAR:PUB, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:RI, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:SAF, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:SAN, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:SGO, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:STM, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:SU, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:TEC, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:UG, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:UL, Bars: 0
Symbol: XPAR:VIE, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:VIV, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:VK, Bars: 2, Last date: 7/15/2013

Provider update complete for Morningstar Static
----------------


QUOTE:
Daily data is being updated...

Updating Provider Morningstar Static ...

Up-to-date symbols: 37, Update required for: 5, New symbols: 0
Symbols already up to date: XPAR:ALU,XPAR:SPI,XPAR:UFF,XPAR:AC,XPAR:ACA,XPAR:AI,XPAR:ALO,XPAR:BN,XPAR:CA,XPAR:CAP,XPAR:CS,XPAR:DG,XPAR:EAD,XPAR:EDF,XPAR:EI,XPAR:EN,XPAR:FP,XPAR:GLE,XPAR:GSZ,XPAR:LG,XPAR:LR,XPAR:MC,XPAR:ML,XPAR:MT,XPAR:OR,XPAR:PUB,XPAR:RI,XPAR:SAF,XPAR:SAN,XPAR:SGO,XPAR:STM,XPAR:SU,XPAR:TEC,XPAR:UG,XPAR:VIE,XPAR:VIV,XPAR:VK,
Symbol: XPAR:BNP, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:RNO, Bars: 2, Last date: 7/15/2013
Symbol: XPAR:FTE, Bars: 0
Symbol: XPAR:PP, Bars: 0
Symbol: XPAR:UL, Bars: 0

Provider update complete for Morningstar Static
----------------


Update completed (4.92 sec)
----------------


This is a nightmare to manage the data with this extension. For quote, it's simple to see if there is a problem of update. But this is impossible to see if the same problem is available with the fundamentals items.

Is it possible for you to check again this extension to try to find a fix ?
profile picture

Eugene

#33
Hi Alexandre,

Thanks for the update log. I was able to nail down the bug in my code. Testing shows that this DataSet can now be reliably updated multiple times w/o issues. The fix is to follow in next update.

P.S. Occasionally you might still see an uncommon "0 bars" when Morningstar actually returns an empty document. You can not combat this - it's a free feed.
profile picture

abegy

#34
Thanks a lot !
profile picture

abegy

#35
Eugene, is it possible to have the Morningstar extension compatible with paper accounts trading ?
profile picture

Eugene

#36
Yes, in upcoming update.
profile picture

Eugene

#37
The Morningstar provider has been upgraded to 2013.09

New: Paper Trading support (requested by abegy)
profile picture

Areality

#38
Does this provider work for anyone? I cannot get any data for AA or any other stock. Below is the log from the Data Manager. Print screen with an empty chart pane attached.

Updating DataSet Test ...
Preparing requests ...
Requests are ready to go.
Updating Security Names for 1 symbols...
Security Names updated.
[0] AA 11010 bars 08.21.2013 0 bars added
Updating Provider Morningstar Fundamental Data ...
AA 83 fundamental items updated
Provider update complete for Morningstar Fundamental Data
----------------

Update completed (4.189 sec)

----------------

profile picture

abegy

#39
You need to add the prefix exchange code before the symbol. For AA, it's XNYS:AA
profile picture

Eugene

#40
Actually, Morningstar Fundamental Data updates without any issues. The prefix is not required, as I seem to get the fundamental data using Yahoo notation (like you just did, because Morningstar provider does not do any "Updating Security Names"):



The issue might be with your system, if the FRED provider fails for you for no apparent reason when it works for everyone else.
profile picture

Eugene

#41
Indeed, the provider has a problem with non-U.S. delimiters. The bug will be fixed in an upcoming release. Thank you for spotting it.

UPDATE

Fixed in v.2013.09.01.
profile picture

Eugene

#42
Areality asked:

QUOTE:
Please add fundamental fields which were presented exclusively in MSN and can be found from Morningstar. Being selfish I need a substitute for "[msn] depreciation" field. It can be found from Morningstar. Here is IBM example http://financials.morningstar.com/cash-flow/cf.html?t=IBM&region=USA&culture=en-US . The third row from top. "Depreciation & amortiz...". Your work is greatly appreciated.
profile picture

Eugene

#43
Thanks for your request. I'll take a look later.

There's twice less data (5 years) for the Cash Flow (Balance Sheet, Income Statement) items than the usual 10 for the "Key Ratio" items, though.
profile picture

Eugene

#44
Areality,
QUOTE:
Being selfish I need a substitute for "[msn] depreciation" field.

I determined that the task of adding is too complex (requires significant effort), and the data is very limited. Considering you already are a YCharts Pro user who have requested to add a depreciation item to that provider, it would be way easier to add a range of depreciation items to the YCharts provider early next month.
profile picture

Areality

#45
Just wondering if anyone at WealthLab looked at Morningstar Premium as an alternative to YCharts Pro. Morningstar data quality seems to be better and the price tag is only $199.00/year vs $1000-$2000 for Ycharts. Maybe it is also a more friendly animal for Wealth Lab developers? Does anyone has experience using Morningstar Premium? How did you like it? I can arrange access to Morningstar Premium for Wealth Lab developer interested in the project.
profile picture

Eugene

#46
The only difference between Free vs. Premium I've found is the data loading depth:

* Quarterly: 5 vs. 10 quarters
* Annual: 5 vs. 10 years

Is it worth it? Do 10 quarters (i.e. less than 3 years) make great difference when compared to the data loading depth at YCharts?
profile picture

abegy

#47
Hi Eugene,

On the Morningstar websute, there is a specific page where you can access a specific market indicator calculated by Moningstar. There are available on the following page (http://www.morningstar.com/market-valuation/market-fair-value-graph.aspx).

Is it possible for you to give us the possibility to download all of them inside the extension ?

Thanks a lot.
profile picture

Eugene

#48
Hi Alexandre,

Sorry, this looks neither promising nor feasible.
profile picture

gbullr

#49
Trying to use Morningstar to pull in revenues.


Used this code.

CODE:
Please log in to see this code.


Pulls in Fiscal Yearly Revenue from 2012.

What am I doing wrong.

Is provider no longer valid?

Thanks.

profile picture

Eugene

#50
The provider is working correctly. For all the Dow 30 symbols I've just tested it on, the last change happened in 2014. Note that the data is annual.

Make sure you're running the latest version 2013.09.01 (see Extension Manager > Others). What were the symbols?
profile picture

gbullr

#51
It was MSFT and it was pulling in


http://financials.morningstar.com/income-statement/is.html?t=MSFT&region=usa&culture=en-US

the 73723 revenue num for June 2012.
profile picture

Eugene

#52
Can not reproduce. MSFT or Dow 30, still works perfectly. Note the 86,833 number for 2014-06. Follow my advice on getting the last version.


profile picture

gbullr

#53
Will re install..

Thanks.

As an aside..

How did you get such a nice yearly chart above?

profile picture

Eugene

#54
QUOTE:
How did you get such a nice yearly chart above?


By switching to Monthly scale. There are two ways as per the User Guide:

+ Charting > Chart Window > "8. Chart Scaling"
+ Charting > Chart Control > Data Panel
profile picture

LenMoz

#55
The Wiki for this provider states, "•The last quarter or trailing twelve months value is dropped.". This does not appear to be true.. When I run the script below for Alcoa, "AA", it shows the last fundamental data item to be the TTM value on the Morningstar web site. It's dated 1/1/2015. I get...
CODE:
Please log in to see this code.
On the Morningstar web site, the 18.4 value is titled "TTM" , and 16.3 is titled "2013-12".


CODE:
Please log in to see this code.
profile picture

Eugene

#56
TTM are no longer being dropped based on customer's request (see messages #27 to #29 in this thread). The Wiki page has been corrected. Thanks for the heads-up.
profile picture

richard1000

#57
Problems in Morningstar Provider:

Updating DataSet Morningstar Dow 30 ...
Up-to-date symbols: 0, Update required for: 30, New symbols: 0
Error setting min/max download threads
Error setting min/max download threads
Update completed (0.125 sec)
profile picture

Eugene

#58
This bug occurs under very rare conditions. I'll upload an updated extension (ETA end of month). Thank you for the heads-up.
profile picture

Eugene

#59
What's new in Morningstar provider's recent version 2016.05?

* Fix: "Error setting min/max download threads" message prevents static provider data update
* Fix: removed artificial limit (.NET default) in New DataSet Wizard preventing from creating DataSets which contain more than ~6500 symbols (32767 chars)
* New: due to holiday bars in EOD data, added support for Market Manager in static data provider
* Change: .NET 4.5 and Wealth-Lab 6.8+ are required to install/update extension
* Change: (internal) updated browser agent version
* Change: (internal) increased thread download timeout to prevent failed requests
* Change: (internal) Multithreaded download routine rewritten. Should improve work on modern multi-core CPUs (e.g. servers)
profile picture

Andrii

#60
Morningstar extentionI I have noticed is using the latest annual financial reportings data versus trailing twelve month, probably I am using it wrong, let us know the way we can switch it to TTM or this is prebuild? Thank you very much in advance.
profile picture

Eugene

#61
The provider returns the data exactly as delivered by the free feed. But don't hesitate to visit the Wiki and search the forums: there's a good chance you'll find a workaround. For example, here's how to get TTM values:

Weath-Lab Wiki FAQ > How can I get TTM, MRQ and FTM values of estimated earnings?
profile picture

Eugene

#62
What's new in Morningstar provider's recent version 2016.11?

NOTE: This update is a must have. It's recommended that you reload all accumulated fundamentals by Morningstar.

* Fix: many values were recognized incorrectly (lower than they are e.g. 2,353 was presented as 2.353 rather than 2353)
* Fix: "Book value per share" is unrecognized due to item renaming in downloaded data
* New: added new items: Payout Ratio %, Interest Coverage, Free Cash Flow Per Share
profile picture

Eugene

#63
What's new in version 2017.04:

* Fix: "Update all data" doesn't update any data during first-time update
* Change: Wealth-Lab 6.9+ required to install/update
profile picture

Eugene

#64
What's new in version 2017.10:

* New: support for automatic bar data corrections
profile picture

tobiastempus

#65
hi,

i dont get fundamental data for european stocks. i read here that there are some Problems with different european ticker Symbols but i dont know how to fix it.

thank you


here is a picture
profile picture

Eugene

#66
Hi,

As for me, the 1st paragraph of the Wiki page you refer to tries to explain it pretty well that the Yahoo! static data for ADS.DE (like shown on your screenshot) cannot be used with the Morningstar fundamental data:

QUOTE:
Since most every data vendor has its own symbology, Morningstar is no different. This is not a big issue for U.S. symbols which are universal, but for overseas symbols it presents a problem. For example, the ticker for a European stock symbol will most likely differ from its counterpart at MSN, Yahoo or QuoteMedia. For example, the same BNP stock traded on XETRA is known as "BNP.F" at Yahoo but exists as "XETR:BNP" at Morningstar. Consequently, attempting to download the historical fundamental data for BNP.F from the vendor that knows the stock as XETR:BMP will fail because it's not recognized.


Solution is obvious. Please visit Morningstar's website and look up the correct ticker for Adidas in their notation (could be XETR:ADS, for example - I didn't check it). Create a DataSet with this symbol using the Morningstar static provider and it should be a good match to Morningstar's fundamental data.
profile picture

tobiastempus

#67
hi,

thank you for your answer.now i get data for adidas .

is there a fast way to create databases for eurostoxx 50,dax30,etc.?or do i have to add every Symbol manually?

profile picture

Eugene

#68
QUOTE:
or do i have to add every Symbol manually?

Yes.

P.S. You can only do that for some other providers, Wealth-Data, Nasdaq Nordic, Dukascopy (to some extent) and Yahoo to name a few. (Nasdaq Nordic doesn't support the named indices and Dukascopy only has a few stocks).

But as for DAX30, Wealth-Data has the cleanest static data possible (although its symbol name e.g. ADS.DE isn't compatible with Morningstar's fundamental data).
profile picture

marcomaier

#69
Hello,
I'm using the Morningstar version 2017.10 and want to use fundamental data for a strategy.
But if I want to use the fundamental data nothing is shown. It seems no fundamental data is existing - see attachment.
Do you know the problem?
profile picture

Eugene

#70
Marco,

I already responded to this very question this month right here. Please re-read my answer in post #66 and the Wiki online guide it refers to. Pretty sure that this will clear your question and pay off in the long run.
profile picture

marcomaier

#71
Hello,

if I use the exact symbol of Morningstar I still don't get fundamental data - see attachments.
profile picture

Eugene

#72
Hi Marco,

Everything works:

1. Check the proper "Morningstar..." fundamental provider (there are two)
2. Update fundamental data using "Update DataSet" (not "Price")
3. Reopen the chart

profile picture

marcomaier

#73
1. Check the proper "Morningstar..." fundamental provider (there are two)

--> I only find one - see attachment
profile picture

Eugene

#74
Good for you, so you're picking the right one and that's not an issue of confusion:

Morningstar Fundamental Provider: extended Dividend data

P.S. Don't open a chart before downloading the fundamental data.
profile picture

marcomaier

#75
Ok, but it doesn't still work ...
profile picture

Eugene

#76
Then you're doing something wrong. I'm pretty sure that it works (see my screenshot above) and the provider has been around for 5 1/2 years.
profile picture

safwanvic

#77
Hi, I'm having problems with updating fundamental data from Morningstar too.

Whenever I update dataset (not price) and reopen the charts, some companies' fundamentals are updated while others remain completely blank. And when I reupdated the dataset, some of the initially blanked companies data may become available, but some which were at first available now become blank.

I double checked with Morningstar website for these individual stocks and confirm the missing data are there. Do you know the problem and how to fix this? Cheers.
profile picture

Eugene

#78
Both the static and fundamental providers work as expected for me. Most likely, your symbol does not have a fundamental data item in the output file. It doesn't matter what Morningstar website shows as the actual export file (parsed by the provider) may have differences. It's not uncommon to encounter empty rows or missing values in this free data source.

If you could be more precise about the symbol and fundamental item then I'd have an idea what went wrong (or maybe it's okay).
profile picture

safwanvic

#79
Sure Eugene. For example, the book value per share for XTSE:CVE was not there. When I reupdated dataset for Morningstar provider it's now available (and it might not be if I reupdate again). While for XTSE:BNS, the data was there. After my post above, I reupdated dataset and it's now missing. The same applies for all fundamental variables, and I scrolled through each company to see availability.

In short, whenever I updated dataset, fundamental data were successfully made available only for some companies. These keep on changing every time dataset is updated (some firms succeed, others don't).
profile picture

Eugene

#80
I updated the two symbols several times and "[ms] Book Value Per Share" has always been present in both. What you describe sounds like a network issue or (more likely) a server-side issue. With a DataSet big enough it's not impossible that your requests are getting throttled or rejected by the M. server. After all this is a free datafeed and web scraping by bots may not be welcome. So you might retry from a different ISP to change your IP address and see if problem persists immediately or later.

P.S. Meanwhile I found a potential issue with the M. static provider which could lead to 0 bars downloaded on some occasions (if download took unexpectedly long). But the M. fundamental provider is altogether different and not affected by this. To be fixed in v2018.12.
profile picture

safwanvic

#81
Thanks, I'm afraid it still doesn't work. The DataSet consists of 60 stocks only. I'm wondering, because when we click on charts WL has this right click "Reload Chart History" function, is it possible to make something like "Reload Fundamental Data" for that particular stock? This way we can try "force" the data to reload on stocks with missing fundamentals.

Re your other info about 0 bars, I have the same problem for the historical price of all ASX stocks which took too long (well, because there are over 2,000 stocks). I would say over 75% constituents returned 0 bars downloaded.
profile picture

Eugene

#82
Safwan, thanks for your detailed feedback and patience to describe the problems. It was very helpful.

The issue with "75% constituents returned 0 bars downloaded" is now fixed. It wasn't that much of an issue with Morningstar before but in my quick test if the download speed slackens the static provider could quit processing prematurely.

The problem with fundamental data didn't present itself on 2 symbols yesterday. When I looked into it closely I was able to reproduce it with a DataSet of your size where up to 50% fundamental data requests ended up empty. Just like I suspected, this behavior of Morningstar's server looks very much like protection against web scraping to me. Without going too deep into detail, even request(s) first in the queue may fail after a pause in download as server responds with empty data at random. :\

Starting with v2018.11 which you already can download (or update to), the fundamental provider (only) will retry up to 5 times if it has encountered no fundamental data. This has proven to be the solution against missing data. (If your DataSet contains symbols not recognized by Morningstar, you'd want to delete them to avoid having to waste a few cycles retrying them 5 times each unconditionally.)

What's new in version 2018.11 - a mandatory upgrade:

* Fix: frequent failure with the static provider (0 bars) if a download took unexpectedly long
* Fix: fundamental provider will retry a failed download on empty fundamental data returned by server at random
profile picture

safwanvic

#83
Thanks a lot Eugene and awesome work for the quick upgrade.

It works much better now. For historical prices of 200 companies the download works fine, while at around 1,000 firms or so I noticed just 1 or 2 missing prices (but simply reloading Chart History fixes this).

For fundamental data, I tested for several sized DataSets a few times: 20 - all fine; 40 - 2/3 missing (reupdate gives all data); 149 (NZ data) - around 40 missing; 200 (ASX data) - 17 missing. You said the fundamental provider will retry up to 5 times if it has encountered no fundamental data, so it will skip if the server doesn't return any after that. Is this cycle purely server-side or affected by my connection speed too? Also, is it possible (maybe in future versions) to update only data for missing symbols, or as per my post above having something like right click "Reload Fundamental Data" on individual stock chart so that we can manually update only stocks with missing fundamentals (without affecting those already available)?
profile picture

Eugene

#84
QUOTE:
It works much better now. For historical prices of 200 companies the download works fine, while at around 1,000 firms or so I noticed just 1 or 2 missing prices (but simply reloading Chart History fixes this).

Was about to suggest this workaround as I noticed partial data returned for a couple symbols during my experiments as well. You can notice them by unusually small Bars count, and I see it by incomplete web pages in my traffic sniffer. But provider cannot make assumptions about how much Bars are 'enough' or 'not enough' so it will not attempt to retry.

QUOTE:
Is this cycle purely server-side or affected by my connection speed too?

Your connection quality or speed is probably not a factor here. Compared to the static data, the footprint of a fundamental data download is nothing. In my opinion, our data download attempts are intentionally counteracted at server side by skipping requests at random. You should get used to it with free data feeds: for example, StreetInsider's protection against bulk capture would now let you get the data for no more than 2-4 symbols before suspension.

QUOTE:
Also, is it possible (maybe in future versions) to update only data for missing symbols, or as per my post above having something like right click "Reload Fundamental Data" on individual stock chart so that we can manually update only stocks with missing fundamentals (without affecting those already available)?

The fundamental provider's design pattern is to reload all data for all participating symbols, and proposed change is undesirable. In my testing on DataSets like Dow 30 or Nasdaq 100, there always were 0 missing fundamental data as the web sniffer detected successful retry attempts. So as a workaround, try to break your bigger DataSets into several smaller ones and update them manually. Fundamental data does not require frequent updates (perhaps once a month).

If this looks inconvenient you might consider paid alternatives like YCharts or Quandl, but they may not deliver the fundamentals for international stocks.
profile picture

safwanvic

#85
Thanks for the detailed feedback.

Noted on the small bars count and possible needs to update as well as the free data feeds limitations. And good idea to break them into smaller DataSets; I can then add tickers with updated fundamentals to a pooled set if testing on a larger portfolio is required.

Indeed not all indices are available on YCharts or Quandl, so far I would say Morningstar is the best in term of wider coverage.
profile picture

Eugene

#86
What's new in v2018.12:

* Fix: potential un-representable date/time error on static data update
profile picture

Eugene

#87
What's new in v2019.09:

* Change: Improved fundamental data update speed by skipping requests for unsupported symbols
* Fix: Fundamental provider sometimes might incorrectly display how many data items were updated
* Fix: Extension Manager icon not shown
This website uses cookies to improve your experience. We'll assume you're ok with that, but you can opt-out if you wish (Read more).