YCharts as fundamental provider choice?
Author: abegy
Creation Date: 10/3/2011 4:21 AM
profile picture

abegy

#1
I would like to know how did you select Ycharts as a candidate to be an extension to Wealth-lab ?

I ask this question becaues I think that Morningstar http://financials.morningstar.com/ratios/r.html?t=IBM&region=USA&culture=en-US is a better one.
It's an international database and you have 10 years online

So, if you look to have a new one provided as an extension, do not forget International Trader :).
profile picture

Eugene

#2
Although it's an interesting source of annual fundamental data items, YCharts beats it hands down.

YCharts has over 65 quarterly fundamental items going back 30 years. Some from their 180+ economic items go back 50+ years. Their export format is convenient and easy to parse.

Please feel free to use the open source fundamental provider, disclosed by your request (among other customers), as a starting point if you're interested in creation of a particular provider.
profile picture

Sabawi

#3
I downloaded and installed the ycharts extension 2011.10. Things went well and the extension manager did not show any update necessary after the initial install. I logged in using my pro account and started a data update for security fundamental data only on a small dataset. Once that completed successfully, I checked the data files at
CODE:
Please log in to see this code.


all the files are there. Now for the real test :

CODE:
Please log in to see this code.


GetFundamentalItem() returned null on all the [yc] indicators I used. So I opened one of the data files in a binary editor and found that the indicators are there but each one is followed by the 4 zero bytes such as "5B 79 63 5D 20 6D 61 72 6B 65 74 5F 63 61 70 00 00 00 00" for "[yc] market_cap 0". Although I did not get any error during that data update, it seems I did not get the data.

Any ideas what went wrong?
profile picture

Eugene

#4
While firing up a binary editor is an interesting idea, first I'd give simple things a go. As the GetFundamentalItem entry in the QuickRef suggests, that "real test" will produce a meaningful result only when the symbol has a fundamental item on the last bar of its data:
CODE:
Please log in to see this code.

Try calling it in the main loop instead.
profile picture

Sabawi

#5

Here is another test
CODE:
Please log in to see this code.


It looks like no data was downloaded
profile picture

Eugene

#6
Yes, no data was downloaded for one or more of the following reasons:

1a. You're not a YCharts Pro customer...
1b. ...or you're not taking a free trial of YCharts Pro.
2. Your YCharts username/password weren't entered in the provider's preferences.

To quote the extension download page:
QUOTE:
Note: the provider requires a paid subscription to YCharts Pro. Before downloading the data, make sure your YCharts username and password were entered correctly.

(Disregard the number of downloaded items the YCharts provider shows in the update log - it's the number of processed calls but not the actual result.)
profile picture

Sabawi

#7
Thanks Eugene -- I am a fully paid pro member on YCHARS, just checked my account page. My next bill is not due for 4 months ;)

It would be nice if there is a check on the download somehow before the log says
CODE:
Please log in to see this code.
profile picture

Sabawi

#8
is there a trace facility I can turn on to at least check that the logon completed successfully?
profile picture

Eugene

#9
If you're a paid customer, then you might want to thank YCharts for finally breaking their data export URL that was working for over a year.

<sarcasm>No surprise though, it almost was in the air. Something had to happen: known for changing data formats every other month, YCharts made no changes since March!</sarcasm>

At first I thought that their server was again returning invalid data like it previously happened on Saturdays, but it's not the case. The provider is currently broken. Thank you for the heads-up. I'll be uploading a fix in a couple of days.

QUOTE:
It would be nice if there is a check on the download somehow before the log says

Unfortunately, YCharts do not have an official API to notify about incorrect password - if you mean to cancel a fundamental data download for that reason. Their server always returns a valid HTML/CSV document, even if it's Error 404 (like today) or there's no data.
profile picture

Sabawi

#10
It is interesting that they do not provide a structured way to validate logons and validate the data!! Does the data come into HTML/CSV only? The reason I am asking is if they provide XML data, we maybe able to avoid the changes in data format.
profile picture

Eugene

#11
Surprisingly, their economic data downloads still work by the old URL. Created a ticket to find out more.
profile picture

Eugene

#12
They provide HTML/CSV only.
profile picture

Sabawi

#13
I just sent them a service request to provide XML data or an API. It does not hurt to ask!!
profile picture

Eugene

#14
It's up to you but the provider is already built upon what's available.
profile picture

Sabawi

#15
Yes Eugene. I am very interested. Please let me know when I can download the fix.

Thanks!!!
profile picture

Eugene

#16
Sorry if I was not clear but my previous comment was regarding XML/API... meaning that if that XML/API appeared, that wasn't enough to rewrite or patch the otherwise working solution. :)

The fix, as noted in another thread today, is coming after YCharts confirms the change is for real and is not a glitch. Don't want to jump the gun.
profile picture

Sabawi

#17
Sorry I misunderstood. Was a bit too optimistic about YCharts fix :) Even if their API appears, it is Not likely to be stable for a while to come. Their support just replied to me saying they are looking into it and wanted some feedback on what I am looking for. That was a good sign. So I think the current extension will still be useful when fixed.
profile picture

Sabawi

#18
Thanks Eugene.

I loaded the fix and updated the data. I ran the second test in this thread. Here is the output:
++++++++++++++++++++++++++++++++++++++++
QUOTE:

AA Item Count: 85
Bar Date Value
-1 4/30/1987 $0.07
-1 7/30/1987 -$0.13
-1 10/30/1987 $0.15
-1 1/31/1988 $0.19
-1 1/31/1989 $0.31
-1 1/31/1991 -$0.33
-1 1/31/1992 -$0.28
-1 4/30/1992 -$1.55
-1 7/30/1992 -$0.07
-1 10/30/1992 $0.07
-1 1/31/1993 -$0.12
-1 4/30/1993 $0.04
-1 7/30/1993 $0.05
-1 10/30/1993 $0.04
-1 1/31/1994 -$0.13
-1 4/30/1994 -$0.01
-1 7/30/1994 $0.12
-1 10/30/1994 $0.14
-1 1/31/1995 $0.15
-1 4/30/1995 $0.27
-1 7/30/1995 $0.31
-1 10/30/1995 $0.32
-1 1/31/1996 $0.21
-1 4/30/1996 $0.25
-1 7/30/1996 $0.19
-1 10/30/1996 $0.10
-1 1/31/1997 $0.20
-1 4/30/1997 $0.23
-1 7/30/1997 $0.30
-1 10/30/1997 $0.33
-1 1/31/1998 $0.31
-1 4/30/1998 $0.31
-1 7/30/1998 $0.31
-1 10/30/1998 $0.30
-1 1/31/1999 $0.30
-1 4/30/1999 $0.30
-1 7/30/1999 $0.32
-1 10/30/1999 $0.35
-1 1/31/2000 $0.43
-1 4/30/2000 $0.47
-1 7/30/2000 $0.47
-1 10/30/2000 $0.42
-1 1/31/2001 $0.45
-1 4/30/2001 $0.46
-1 7/30/2001 $0.35
-1 10/30/2001 $0.39
-1 1/31/2002 -$0.15
-1 4/30/2002 $0.26
-1 7/30/2002 $0.27
-1 10/30/2002 $0.23
-1 1/31/2003 -$0.27
-1 4/30/2003 $0.17
-1 7/30/2003 $0.26
-1 10/30/2003 $0.33
-1 1/31/2004 $0.32
-1 4/30/2004 $0.41
-1 7/30/2004 $0.46
-1 10/30/2004 $0.32
-1 1/31/2005 $0.30
-1 4/30/2005 $0.30
-1 7/30/2005 $0.52
-1 10/30/2005 $0.33
-1 1/31/2006 $0.25
-1 4/30/2006 $0.69
-1 7/30/2006 $0.85
-1 10/30/2006 $0.61
-1 1/31/2007 $0.42
-1 4/30/2007 $0.75
-1 7/30/2007 $0.81
-1 10/30/2007 $0.63
-1 1/31/2008 $0.75
-1 4/30/2008 $0.37
-1 7/30/2008 $0.66
-1 10/30/2008 $0.33
-1 1/31/2009 -$1.46
-1 4/30/2009 -$0.61
-1 7/30/2009 -$0.47
-1 10/30/2009 $0.08
-1 1/31/2010 -$0.23
-1 4/30/2010 -$0.20
-1 7/30/2010 $0.13
2 10/30/2010 $0.06
64 1/31/2011 $0.25
127 4/30/2011 $0.27
190 7/30/2011 $0.28


It is working!!

Notice the last eps reported in October 2011 was not included.

I will keep testing it with the other values.

Again, thank you very much!!
profile picture

Eugene

#19
You're welcome. Glad I could help.
QUOTE:
Notice the last eps reported in October 2011 was not included.

You might want to review a conversation in the primary thread on YCharts integration to understand how the fundamental data items are timestamped and why.
profile picture

Sabawi

#20
Hi

It seems that YCharts data is broken again. It appears it was broken after Jan 12 2012. Can you please check if they have changed their format again. Thanks!!!
profile picture

Eugene

#21
YCharts have recently notified us about an ongoing change to their data export. The provider will be updated to reflect the change (ETA = early next month).
profile picture

Eugene

#22
Fixed the YCharts data export URL change.

Everyone using YCharts should update to v2012.02 to continue using their service.
profile picture

catweazle

#23
Could it be that the Ycharts download interface is broken again?

I run WL Pro 6.4 64-bit, I have an active Ycharts Pro Gold membership and the 2012.6.0.0 version of the Ycharts data provider. My email and password are entered (correctly, I checked).

When I try to update the fundamental data, I see the following:
CODE:
Please log in to see this code.


But when I try to visualize the data (say, dragging the [yc] assets fundamental item onto an AA chart, it shows nothing. Running Sabawi's last code snippet confirms the suspicion. Nothing is downloaded.
profile picture

Eugene

#24
Yes, it could be broken again. Something is used to break there every several months.

We do not maintain a YCharts subscription, and YCharts did not respond to our inquiry on providing a free account for the purposes of development and support of WL+YCharts customers. If you want us to take a look at it, please consider sending a letter to info@wealth-lab.com with your email and YCharts password. After our troubleshooting is finished, you'll simply change the password. This is how the latest fix (2012.06) was made.
profile picture

catweazle

#25
Wow, that was fast. Thanks, Eugene. I sent you an email.
profile picture

Eugene

#26
Thanks for your assistance. Indeed, it turned out to be yet another data export URL change (fourth on my memory). Will keep everyone posted.

Edit

The problem has been fixed. Unfortunately, we can't upload updated extensions on our website at this time. Check back soon.
profile picture

Eugene

#27
YCharts has become even a better fundamental provider choice - with limited set of free data.

For more details please see:

http://www2.wealth-lab.com/WL5WIKI/YChartsFundamental.ashx
http://www.wealth-lab.com/Forum/Posts/YCharts-Fundamental-Data-for-Securities-and-Economic-Data-provider-30761
profile picture

Areality

#28
Thanks for adding bunch of Depreciation fields, do not need that many actually) End of Quarter fields would be just fine, others can be calculated within the Wealth Lab.

Have 3 more fields to add if possible:
total_long_term_assets (quarterly)
long_term_investments (quarterly)
working_capital (quarterly)

Probably would not hurt anyone if you add them at the expense of some secondary Depreciation fields (i.e. Depreciation Growth, Depreciation per share etc). I am just worrying about Ycharts Exports quota, since every additional field increases the traffic.

profile picture

Eugene

#29
Request has been added to product backlog for future consideration.
profile picture

Areality

#30
Thanks. Yet another one:

Changes in Working Capital (Quarterly) - http://ycharts.com/companies/AAPL/total_changes_in_assets_liab

In bold the field name as used by Ycharts

Actually you can replace previously requested working_capital (quarterly) with this one. Was hoping to get change in working capital using the raw data, but after some additional number cruncing realized that it is not possible.
profile picture

Eugene

#31
Noted.
profile picture

Eugene

#32
QUOTE:
Probably would not hurt anyone if you add them at the expense of some secondary Depreciation fields

You raised a very interesting point.

To every YCharts user out there:

YCharts has a tight download quota for Pro accounts ($99) of only 25Mb/month. Our users tend to run out of the quota pretty soon. As a relief, we can drop some unneeded fundamental and economic data items, saving traffic and increasing download speed.

Please tell us which fundamental and economic items you are using? and which you never used? which ones must be kept and which may be dropped.

Everyone please review the complete list and vote here on the forum:

http://www2.wealth-lab.com/WL5WIKI/YChartsFundamental.ashx
profile picture

Eugene

#33
To all YCharts Advanced / Professional subscribers out there: does economic data download work for you or did they broke?

(Free economic data downloads are broken but this will be was fixed. So I'm interested in hearing from paid subscribers only, as they use a different mechanism for downloading the data.)