Quandl static provider + Zacks fundamental provider
Author: Eugene
Creation Date: 11/30/2014 6:55 AM
profile picture

Eugene

#1
Just released: a suite comprised of Quandl static data provider and Zacks fundamental data provider. With its help, Wealth-Lab users can download countless datasets from Quandl website, such as:

* free and paid static EOD data for global instruments (futures, commodities, rates, stocks, indices, bonds, currencies...)
* COT data (in legacy and new formats)
* subscription-based fundamental data by Zacks (dividends, earnings esimates/surprises, analyst ratings)


The provider is available for installation from our website (Extensions section):

Quandl provider - installation link

Please read the online setup guide before using:

Quandl provider online notes in the Wealth-Lab Wiki


Currently, we support futures from OFDP and CHRIS groups, COT data, stocks (although you wouldn't want them as Quandl takes the data off of Yahoo and Google). If a symbol is unsupported or returns empty data, you've come to the right place.

Please suggest your symbols and report bugs here!
profile picture

Eugene

#2
Happy New Year!

Version 2015.01 is out with a fix which is a must have for most international customers: fixed incorrect data in some locales.
profile picture

shardis89

#3
Apologies if this is not the spot to post this, but in the Quandl provider online notes in the Wealth-Lab Wiki link from above, how would I arrive at defining the Bars.NamedSeries for a specific COT Data named series? Better yet, how could I create a custom dataseries from the COT Named Series data defined as Commercial Long - Commercial Short and then apply an indicator (e.g. RSI.Series(Close,2)) to the custom dataseries to normalize the custom index. Hence we are creating a Commitment of Traders Index indicator without coding a bunch of math, but rather utilizing the pre-programmed indicators to do the calculations on the custom dataseries.
profile picture

Eugene

#4
I added the code sample that you requested to the Wiki link above. Note that it can only be used with Quandl symbols containing legacy series i.e. ending with "_L_ALL" (L is for legacy):

Quandl provider suite > COT Data (scroll to the very bottom of the page)
profile picture

shardis89

#5
Awesome, thanks Eugene.

Since the idea is to use the data as an indicator and visualize how it relates to price, I added to it so that a person could pull in the Quandl Futures Price data for the related COT data. Since the COT data is compiled weekly, in Wealth Lab you have to set the scale to daily. So when pulling in the futures data you must set that data to synchronize to the weekly data manually in the code. This code also highlights the overbought and oversold conditions for all the visable panes. Below is the code I used.

CODE:
Please log in to see this code.
profile picture

lukeallen63

#6
Request for new data collection from Quandl...

I ran across a premium paid data collection called Steven's Continuous Futures (SCF). I haven't subscribed but would be interested in using the data since they cover a lot of futures and scrub the data for errors. I'm not sure something would need to be setup to use it or not though, if anyone knows please advise.

Thanks!
profile picture

lukeallen63

#7
Just noticed this on the ODFP section of Quandl...

QUOTE:
"OFDP is being deprecated as a database on Quandl in favour of more comprehensive databases. All the futures data, and more, under OFDP can be found for free under exchange databases, CME, ICE, LIFFE, EUREX, and continuous contracts can be found under CHRIS. We are sorry for any inconvenience this may cause. If you have any questions please don't hesitate to contact us at connect@quandl.com ."


I can also see provider's like the CME group, so maybe adding some of those as well will help users get futures data.

I personally would find this very helpful

Thanks again
profile picture

Eugene

#8
QUOTE:
Request for new data collection from Quandl...

Simply give it a try as explained in provider's online guide in the Wiki and you'll see that SCF is already supported (for trial users too). Don't forget to type in your API key. Most data sources, whether existing or new, are picked up automatically. See "How it works" for more details on automatic format detection. A request is needed only if something doesn't work.
profile picture

lukeallen63

#9
Thanks Eugene, I did try it and it worked. Thanks again, this is a great tool.
profile picture

sedelstein

#10
I read the Wiki http://www2.wealth-lab.com/WL5WIKI/ZacksFundamental.ashx

Right now, there are 4 data items available

QUOTE:
Out of fundamental items available on subscription basis, four are currently supported:

[z] Zacks_Analyst_Ratings
[z] Zacks_Dividend
[z] Zacks_Earnings_Estimates
[z] Zacks_Earnings_Surprises


Zacks/Quandl offer many more. Is there way to get them through WL if you are a Zacks subscriber?

Thanks for your help
profile picture

Eugene

#11
Many more is probably not the correct word. Zacks Street Earnings Estimates, Consensus Earnings Trends, Sales Surprises seem somewhat less popular to our audience to justify their possible development at the moment. The two remaining mainstream collections, Zacks Fundamentals Collection A and B, may be of potential interest for development due to returning 270+ fundamental items for $3,000/year. This task is on our product backlog (low priority). If there's enough community following, this can help prioritize it. For now, you can access a lot of fundamental items with a subscription to YCharts, for example.
profile picture

sedelstein

#12
Thanks Eugene

Fundamentals A&B were what I was thinking of.

YCharts Pro is $300/month so Zacks is marginally cheaper and looks to have orders of magnitude more fundamental items

Hopefully others read this post and chime in.
profile picture

Eugene

#13
Please note that a valid Quandl API token (customer, full access) will be required in order to develop the provider. The demo key allows to make only 100 requests which is not enough to backfill even a single symbol once. This is a must. Working on it with the Quandl team.

QUOTE:
YCharts Pro is $300/month so Zacks is marginally cheaper and looks to have orders of magnitude more fundamental items

It's off topic here but to close this discussion, both points seem arguable:

1 - There's a $40/month plan (however, seriously limited to a 25MB/month data export quota).
2 - It's possible to ask us to implement items available on their website but missing in the provider.
profile picture

mikesblack

#14
Question about Zachs Data. After update I am only finding one item, last date is the present date for Zachs analyst ratings and earning estimates. Earning surprises go back till 6 2013 and Dividends go back until 1 1992. I sampled a number of stocks with the same results. What will I need to do to acquire more data from Zachs?
profile picture

Eugene

#15
It's better if you inquire Quandl or Zacks support on this. If you are a subscriber, there's premium support. For demo users, serious restrictions apply - like making only 100 data requests.
profile picture

mikesblack

#16
Thanks Eugene. I'll check it out.

What does Quandl offer that Zach doesn't and visa versa?
profile picture

Eugene

#17
Quandl is an aggregator that provides access to various vendors, Zacks is one of them.
profile picture

mikesblack

#18
Thanks!

Eugene, Can you explain the process of how I can download the Quandl data? If I go to their site, there are many data sets, free and premium. How do I add them into the Data Manager?
profile picture

Eugene

#19
I realized that this question may be coming and this is why the very first post in this thread has a help link in bold. You may have noticed that we maintain documentation for most providers in the Wiki. Quandl is no exception. Scroll up and you shall find the online guide with your question answered.

P.S. Quandl is like a shopping mall with lots of independent boutiques. With the various types of data hosted by Quandl, obviously not every data feed is compatible but the provider is smart enough to figure out most types of data it is processing (like COT, stocks, futures, rates etc.)
profile picture

mikesblack

#20
Thanks Eugene. Very helpful. Have a great weekend.

Aside from what is already provided, if I purchase one of their services, for example Zack analyst rankings, how would I access that data and then display it on a chart,etc.?
profile picture

Eugene

#21
Mike, if I say something like "You may have noticed that we maintain documentation for most providers in the Wiki", you can trust me on that. It was only yesterday and nothing has changed:

Zacks fundamental provider

By the way, this link could be found in the first paragraph of the Quandl page I referred to in my penultimate reply.

Re: display it on the chart. Since the approach is universal for every fundamental provider, you can find the answer in the Wealth-Lab User Guide.
profile picture

mikesblack

#22
Sorry Eugene, I did read the Wiki page and have set up the Data Manager as illustrated. Just unclear how to make use of a payed service. Zacks didn't have a clue about WealthLab integration. Quandl have tons of cool packages, but I am still unclear how to access or update the data. I wouldn't want to pay for a service I can't put to immediate use.
profile picture

Eugene

#23
That's why there are tons of free data to play around with and get a feel of the Quandl provider, a (very limited) free trial of those cool packages (paid services), and two pages of documentation covering everything you need (hopefully).
profile picture

kbellare

#24
I'm getting 0 data results (scresnshot below) when downloading from Quandl - CHRIS dataset. I installed Quandl (with my API key), added 41 symbols to the dataset and tried downloading. Is there something i'm missing? How do i test that the Quandl service is sending data?

profile picture

Eugene

#25
All these symbols are invalid. To save yourself time, visit Quandl's website and verify that none of these tickers exist (I did). From a logical standpoint, this is what I would do with any given data provider. Now you've learnt this trick, too.

Bonus:

Add a "2" to each ticker and it will return data e.g. CHRIS/CME_AD2. (Don't ask me why, ask Quandl.)
profile picture

Dave B.

#26
Eugene:

As an early adopter of Quandl and having brought it to your attention, I thought you and the Wealth-Lab Community might be interested in this article just published:

https://www.quandl.com/blog/alternative-data

For the record, I have no connection to Quandl.
profile picture

Eugene

#27
What's new in version 2016.08:

* Fixed no data for CBOE/VIX symbol
* Removed artificial limit (by default) in New DataSet Wizard preventing from creating DataSets which contain more than ~6500 symbols (32767 chars)
profile picture

Eugene

#28
What's new in version 2016.09:

* Added support for CBOE/VVIX symbol (requested by ReneW)
profile picture

Panache

#29
I'm doing something wrong, but I don't know what it is.

My API Key is entered in the Quandl tab. I created a Quandl data set containing "WIKI/A". When I Update Data, I just get:

Updating Provider Quandl ...
Provider update complete for Quandl
----------------

Update completed (0.0 sec)
----------------

I can get the data in Python, so I know my API Key and symbol are correct. I can also see the data at https://www.quandl.com/data/WIKI/A-Agilent-Technologies-Inc-A-Prices-Dividends-Splits-and-Trading-Volume

The QuandlStaticProvider directory is empty except for QuandlSettings.xml (which does contain my API Key).

Any ideas?
profile picture

Eugene

#30
I think you ran into an intermittent first-time data update issue. As a workaround, I suggest that before trying to "Update All Data" after first-time DataSet creation by a new provider (e.g. Quandl), you'd either make a one-time update of the Quandl DataSet or restart WLP.

Why would Data Manager not download and update any data?

Please let me know if it's helpful or not.
profile picture

Panache

#31
Thanks Eugene.

Restarting Wealth-Lab Pro didn't help, but clicking "Update DataSet" solved the problem. After doing that, "Update all data for selected Providers" works as expected.

I just want to point out to anyone else reading this thread that all Quandl stock symbols must contain the prefix "WIKI/", which you must add to each of the exchange symbols, ie."A" is "WIKI/A". As a result, Wealth-Lab Pro puts all stock symbols in the Quandl/Daily/W directory. Therefore, I if your Strategy uses the exchange symbol, you have to add "WIKI/" to the symbol before calling methods such as GetExternalSymbol.
profile picture

Eugene

#32
QUOTE:
all Quandl stock symbols must contain the prefix "WIKI/"

That's correct as long as you're downloading free EOD stock data from the "Wiki EOD Stock Prices". For every other (currently only paid) databases there's a different prefix.

QUOTE:
Restarting Wealth-Lab Pro didn't help, but clicking "Update DataSet" solved the problem.

Thanks for the update Kurt. I'll revisit this issue.
profile picture

Eugene

#33
QUOTE:
Restarting Wealth-Lab Pro didn't help, but clicking "Update DataSet" solved the problem. After doing that, "Update all data for selected Providers" works as expected.

According to my findings, this is a limitation/issue in Wealth-Lab. Most providers using the BarDataStore are affected - in particular: FRED, Dukascopy, Quandl and Google providers. However, it's pretty easy to work around on our end. A fix is coming in next extension update (2017.04).
profile picture

Eugene

#34
@Panache: fixed, please update the extension and restart WLP.

What's new in version 2017.04:

* Fix: "Update all data" doesn't update any data during first-time update
* Change: Wealth-Lab 6.9+ required to install/update
profile picture

pnperez

#35
Hello

I used the Quandl data static provider for some time but since mid april the display of graphs in wealth lab is wrong
i use this database SCF continuous futures
the data i capture is correct but the graphs are wrong

https://www.quandl.com/data/SCF-Continuous-Futures
I use price data ending in FW

if i do copy to clipboard data, there is no doubled up price data, and there are none on the quandl website if i download manually
could you check if there is a bug in the graph display?

Thanks in advance
profile picture

pnperez

#36
Hello

I used the Quandl data static provider for some time but since mid april the display of graphs in wealth lab is wrong
i use this database SCF continuous futures
the data i capture is correct but the graphs are wrong

https://www.quandl.com/data/SCF-Continuous-Futures
I use price data ending in FW

if i do copy to clipboard data, there is no doubled up price data, and there are none on the quandl website if i download manually
could you check if there is a bug in the graph display?

I attach the jpg of the print screen

Thanks in advance
profile picture

Eugene

#37
Hello,

I don't think there's anything wrong with the provider as I use Quandl data too and would've noticed that. Probably your data is corrupt. Right click on the chart and select "Reload Chart History" to reload the symbol from scratch.

The free sample data (2517 bars ending in 2014) loads up correctly for me as can be seen (please use PNG instead of JPG for screen captures):

profile picture

pnperez

#38
Hi Eugene

Yes it works if I reload the chart

Thanks for your help

Pierre
profile picture

Eugene

#39
Pierre, that's good news to hear! Glad to help.
profile picture

Eugene

#40
What's new in version 2017.06:

* New: AAII sentiment data (requested by richard1000)
profile picture

Eugene

#41
What's new in version 2017.08:

* New: automatic recognition of any data having just 2 fields as Close-only data
profile picture

Eugene

#42
Changes in version 2017.09:

* Fix: increased stability by fixing startup crash on broken configuration file
profile picture

Eugene

#43
I was thinking about creating a data provider for Bitcoin data but stumbled onto a Quandl ticker for Bitcoin price at Mt.Gox. Simply add BCHAIN/MKPRU to your Quandl DataSet(s) to track Bitcoin in Wealth-Lab:

BCHAIN/MKPRU

Just a proof how comprehensive has Quandl become.

UPDATE 10/14/2017: Another bitcoin dataset discovered by a Wealth-Lab community user:

BCHARTS/BITSTAMPUSD
profile picture

Aldo

#44
Hi,

please is there a way to attribute a SecurityName to Quandl Symbols (or any other symbol missing a name)?
For example
"Gold" for CHRIS/CME_GC1
"Bund" for EUREX_FGBL1 (!)

"Symbol info manager" is the tool that I expected could be used for this, but there is no SecurityName field there.

It is apparently a minor problem, but apart from identifying odd symbols, a customizable name could also be useful to sort securities within datasets -- if of course datasets panel would display names instead of symbols.

Thanks in advance for the answer.

Kind regards
Aldo
profile picture

Eugene

#45
Hi Aldo,

There is no easy workaround to append a secuity name to existing data.

While some of Quandl's APIs may return company name, this isn't the case for the API our parser is based on (unfortunately). We have this enhancement idea in our product backlog as a low-priority task but I'm making no promises currently: it's not clear if it's feasible yet.
profile picture

Eugene

#46
Good news Aldo. Starting from v2017.11, the Quandl provider will append security names to new symbols. This is to minimize the number of data requests since we're going to add an extra request for security name. For the security name to appear in an existing symbol you'd have to remove all its data (e.g. using "Remove all data" feature of the Data Tool extension) and update the DataSet (or "Reload chart history" if you don't have that much symbols).
profile picture

Aldo

#47
Hi Eugene,

thanks, that´s great, I´ll wait for the new version to be released then.