Cryptocurrency (Bitcoin) data provider
Author: GolfPlayer
Creation Date: 10/15/2017 5:48 PM
profile picture

GolfPlayer

#1
Do you have a provider which gives crypto currencies data?
profile picture

Eugene

#2
We will have one soon, to be released late October - early November. While you're waiting, see my post #43 in this thread: Quandl static provider + Zacks fundamental provider. Quandl is a universal data marketplace and two Bitcoin datasets are already available for use in WLD/P for free.
profile picture

GolfPlayer

#3
Thanks!!
profile picture

Eugene

#4
Meet the first Wealth-Lab data provider that downloads historical daily and hourly data for cryptocurrencies (read: Bitcoin) and their cross rates from Cryptocompare.

* Direct installation link
* Online user guide in the Wiki

Provider comes preconfigured with some top coins for a quick start: Bitcoin, Ethereum, Litecoin, Monero etc. Make sure you've read the online manual on how to create a valid coin symbol that will be recognized by the provider!
profile picture

Eugene

#5
What's new in 2017.12?

* New: ability to enter symbol names without the base currency. For example, if you simply enter BTC: provider will automatically translate this to BTC/USD. Same for all other recognized symbol names. BTC/EUR or USD/BTC will still be correctly recognized as Bitcoin/Euro or Dollar/Bitcoin.
* New: added Bitcoin Cash to default preset of New DataSet Wizard
* Fix: Data failed to update after weekends in some cases (provider wrongly indicated that symbols are up-to-date)
profile picture

Eugene

#6
As an alternative take a look at the CBOE provider for EOD data of exchange-traded Bitcoin futures (XBT):

CBOE Put/Call Ratio and Bitcoin futures static data provider - installation link
CBOE Put/Call Ratio and Bitcoin futures static data provider - forum thread
profile picture

Eugene

#7
Regarding the CryptoCurrency provider. An initial update of a 60-Minute DataSet w/ multiple symbols may end up with 169 bars returned for every symbol. Looks like a possible bug or restriction in this API. As a workaround I suggest to perform initial update with just one symbol doing a "modify DataSet + Update DataSet" until you collect the data for all your cryptocurrencies. I'll look into working around this inconvenience in a future build.
profile picture

maninjapan

#8
I followed the instructions as above and was still only able to get 169 bars for 60 minute data. I tried a few times switching between a single symbol at a time of BTC and ETH. My understanding is that the above is a work around to get more than 169 bars, is this correct?
profile picture

Eugene

#9
Have you seen my post #7 above? If not, please review it.
profile picture

maninjapan

#10
Yes, that is what I was referring to. I did as follows

1. Create new Data Set, input 1 symbol only, BTC, bar scale = Hourly
2. Update Data

Data Log is as follows. Have I misunderstood the instructions? The above is how they are explained in your comment and the wiki as I understand them..... ( I also tried modiifying the symbol to ETH and then updating data again, but still only 169 bars)


Updating Provider Cryptocurrency (Bitcoin) ...
60 Minute data is being updated...

Up-to-date symbols: 0, Update required for: 1, New symbols: 0
Symbol: BTC, Bars: 169, Bars corrected: 1, Last date: 2/27/2018

Provider update complete for Cryptocurrency (Bitcoin)
----------------

Update completed (2.106 sec)
profile picture

Eugene

#11
The real point of my message was this:
QUOTE:
I'll look into working around this inconvenience in a future build.

And the wait is not too long ;)
profile picture

Eugene

#12
What's new in v2018.03:

* Fix: occasional error message when updating newly added currency symbols
* Fix: attempt to alleviate the crippled Hourly data returned after making several big requests
profile picture

maninjapan

#13
Thanks Eugene, I have downloaded the update and can confirm there is improvement (for any others interested). Am now able to download 2001 bars on the hourly data. Just to confirm though. For now is this the maximum amount we can expect to download for a single symbol? ( I may have misunderstood your above workaround to download more data for a single symbol at a time).

Is it possible to select periods of data to download? ie, not just most recent 2000 bars?

Thanks again!!
profile picture

Eugene

#14
Deane, here's an update log from minutes ago for a bunch of top cryptos which I've downloaded from scratch:

Updating DataSet Cryptocurrency (Minute,60) ...
Up-to-date symbols: 0, Update required for: 0, New symbols: 6
Updating new symbols...
Symbol: BTC, Bars: 43550, Last date: 28.02.2018
Symbol: DASH, Bars: 35543, Last date: 28.02.2018
Symbol: ETH, Bars: 22464, Last date: 28.02.2018
Symbol: LTC, Bars: 38112, Last date: 28.02.2018
Symbol: XMR, Bars: 27023, Last date: 28.02.2018
Symbol: XRP, Bars: 27230, Last date: 28.02.2018
Update completed (1 min 50.294 sec)
----------------

For me it works in v2018.03 - as can be seen I get the entire history from day zero. 2001 bars means the website is throttling you. It's the same behavior as getting 169 bars before. Please don't ask me: I don't know what's causing it on their end. Quandl, Tiingo, YCharts, Cryptocompare... every other data provider these days has quotas or a throttling policy (which isn't always public).

Just a couple of closing suggestions. Make sure on demand update is turned off in the Data Manager. Don't 'abuse' their API by running streaming charts or issuing repeated requests. Retry from a different IP address etc. And finally, consider using EOD data - this seems to be less affected.
profile picture

maninjapan

#15
Thanks for the clarification Eugene, appears to be a problem between myself and them then.

I tried with LTC and got 38,112, 5 minutes later tried ETH got 2001... ( I don't use streaming charts). I will try from a different IP address (EOD data appears to be fine). I will try again later

Very much appreciate the timely support though!

Thanks
profile picture

Eugene

#16
My pleasure. By the way, there isn't a Streaming provider for cryptocurrencies (nor I plan to build it) - sorry for the confusion. :)
profile picture

maninjapan

#17
Just a couple of things Ive noted while playing with this for a while for other users to keep in mind. Eugene appears to be able to download without issue so there appears to be factors outside of Wealth lab affecting how much a user will be able to download.


As Eugene states, daily data seems to have fewer issues with download limits than intraday

Regardless of how I try to download data (including one symbol at a time or small groups at a time, leaving time between downloads) BTC,LTC,ETH will only give me 2001 bars while other symbols will give me a fuller history ( XRP 27,254 , BTC/JPY 57,079, NEO, 5000)

profile picture

Eugene

#18
I think what I'm about to say is evident but let's still stress it. Once you've downloaded 2001 bars in a symbol (let's say as a consequence of throttling), subsequent updates will not bring you more data than 2001 bars. That's because its history has been backfilled. To get more data you will have to delete the accumulated data:

1. for many cryptos: by using the Data Tool's "Remove all data" feature and then updating your crypto DataSet
2. for a handful of symbols: by right-clicking its chart and selecting "Reload chart history".
profile picture

Eugene

#19
What's new in v2018.05:

It's strongly advised to upgrade to this build to fix a data accuracy issue caused by the server:

* Fix: bogus zero bars may be prepended to the data by Cryptocompare's API on data update.

It's recommended to reload your Cryptocurrency provider's data from scratch if it has bars before the date when symbols have started trading.