AlphaVantage provider: free historical intraday/daily data for U.S. stocks and Forex
Author: Eugene
Creation Date: 1/28/2018 11:05 AM
profile picture

Eugene

#1
I had considered building a data provider suite for the AlphaVantage feed but discarded the idea due to not enough benefits compared to the already available data providers.

There's no shortage of EOD data connectors for Wealth-Lab but Alphavantage returns intraday historical data. However, the free intraday history spans only the last 10-15 trading days which is no better that the Google provider already offers.

Some of its users reported slowness. Another restriction is making 200 calls per minute. On the whole, I'm not considering investing my effort into it at this time.
profile picture

Eugene

#2
As the Google data feed has been decommissioned, there's a gap to fill. Just released: AlphaVantage data provider! You can get free intraday (limited) and daily data for U.S. stocks and Forex.

1. Install the provider and restart WLP/D: Direct installation link

2. Sign up at https://www.alphavantage.co for a free API Key, copy and paste it to Wealth-Lab's Data Manager > AlphaVantage tab.

Close it for the changes to apply.

3. Check out the Wiki for its Online user guide. Read "Issues and Limitations" before reporting any.

Start getting free intraday and daily data for U.S. stocks and Forex!
profile picture

Eugene

#3
What's new in v2018.12:

Fix: potential error on data update (The added or subtracted value results in an un-representable DateTime).
profile picture

Harapa

#4
Is this data provide defunct?. Last bar I get is dated in 2019. Same for Intraday data. Plus no data for Equity symbols. My API key is valid and recognized at their web site.

[2] .NDX 4889 bars 6/10/2019 0 bars added
[0] .DJI 4889 bars 6/10/2019 0 bars added
[0] .SPX 4880 bars 5/28/2019 0 bars added
[4] .VIX 4880 bars 5/28/2019 0 bars added
[[1] AAPL Error: No data
[0] EEM Error: No data
[3] DIA Error: No data
[4] GDX Error: No data
[2] GLD Error: No data
profile picture

Eugene

#5
We employ a watchdog that alerts if any data scraper in the wild (all data providers, their Classification Groups, C.Components, public Strategies that downloads data on-the-fly, countless other utility scripts etc.) goes down or starts returning unexpected results. Such automation saves a lot of time and effort and lets me act proactively.

No, AlphaVantage is alive and working. You should check with the data vendor on what's included in your subscription as we have no information.

Updating DataSet AlphaVantage (Minute,1) ...
[0] AAPL 2851 bars 09.07.2020 2851 bars added
Update completed (4.925 sec)
----------------

Your account might have ran out of quota, for example.
profile picture

Harapa

#6
OK, so I managed to update in batches of 3-5 symbols for the initial run. But data includes extended hours. I have all symbols added in Market Manager, provider selected, and market hours selected as 09:30-16:00 EST. Shouldn't this limit the data series to the hours indicated only (exclude pre and after hour data)?
profile picture

Eugene

#7
Yes it should and does if configured properly. That happens because you did something incorrectly. I have no doubt that enabling AlphaVantage in the MM and putting symbols in the US Equities list works. Make sure no other group with different settings 'intercepts' your symbols. Mark the US Equities as the default market group.
profile picture

rmandel00

#8
Harapa,
I signed up for AlphaVantage, got the API and uploaded some symbols on a daily dataset in WL. It was unable to load the volatility etf's in the form of VIX or ^VIX or VXN or ^VXN. Any thoughts on these symbols? Are they not available? Other ETF symbols like QID and QLD load and populate.
Thanks,
rmandel00
profile picture

Eugene

#9
VXN and VIX aren't recognized as valid symbols. AlphaVantage is about 1) stocks and 2) cryptos. There are many other data providers supporting these symbols. Here's map of our data providers - feel free to ask questions in respective forum topics:

Map of Wealth-Lab Data Providers
profile picture

Harapa

#10
@rmande100
.VIX is the only supported symbol
QUOTE:
Updating DataSet AlphaVantage (Minute,5)b ...
[4] .VIX9D Error: No data
[0] .RVX Error: No data
[2] .VXMT Error: No data
[3] .VIX3M Error: No data
[4] .VXN Error: No data
[0] .VXV Error: No data
[1] .VIX 319662 bars 7/17/2020 0 bars added
Update completed (3.368 sec)
profile picture

Harapa

#11
Vendor is not a reliable source of updates (or may be I am doing something wrong). It updates when it wishes to. Here is the log of this AM update. BTW, adding longer history file is not the reason.Similar behavior noted for other time frames.
QUOTE:

Updating DataSet AlphaVantage (Minute,5) ...

[0] .DJI 322710 bars 7/16/2020 0 bars added
[2] .SPX 322714 bars 7/16/2020 0 bars added
[1] .NDX 323058 bars 7/16/2020 0 bars added
[3] .VIX 319736 bars 7/20/2020 74 bars added 85 bars corrected
[0] DIA 318805 bars 7/20/2020 151 bars added
[4] AAPL 399070 bars 7/20/2020 170 bars added
[1] IWM 319708 bars 7/20/2020 166 bars added
[3] JNK 22749 bars 7/17/2020 0 bars added
[2] GLD 398800 bars 7/20/2020 152 bars added
[0] MDY 397653 bars 7/16/2020 0 bars added
[4] QQQ 319566 bars 7/16/2020 0 bars added
[1] SDS 319373 bars 7/17/2020 0 bars added
[3] SH 319207 bars 7/17/2020 0 bars added
[2] SPHB 54118 bars 7/17/2020 0 bars added
[0] SPLV 175473 bars 7/17/2020 0 bars added
[4] SPXL 319531 bars 7/17/2020 0 bars added
[1] SPXS 318333 bars 7/16/2020 0 bars added
[3] SPY 398867 bars 7/16/2020 0 bars added
[2] SSO 318326 bars 7/16/2020 0 bars added
[0] SVXY 397994 bars 7/16/2020 0 bars added
[4] TLT 314491 bars 7/16/2020 0 bars added
[1] UNG 259762 bars 7/20/2020 796 bars added 700 bars corrected
[3] UVXY 320463 bars 7/20/2020 192 bars added
[2] VXX 318484 bars 7/20/2020 74 bars added 1 bars corrected
[0] VXZ 318446 bars 7/20/2020 12 bars added 1 bars corrected
[1] XLP 315001 bars 7/17/2020 0 bars added
[4] XLF 318083 bars 7/20/2020 74 bars added 1090 bars corrected
[3] XLY 397803 bars 7/17/2020 0 bars added
Update completed (1 min 31.884 sec)
----------------
profile picture

Eugene

#12
Online user guide for AlphaVantage static data provider > Issues and Limitations:

QUOTE:
* Check out the feed's website for usage quota/limitations on the free tier (e.g. 5 requests per minute) or premium plans.
profile picture

Harapa

#13
For free tier building a 1 min delay between 2 batches of 5 symbols will be a good feature to have!
profile picture

Eugene

#14
It doesn't look possible to tell programmatically if you're a free user or on paid subscription. A choice between Free/Paid (think radio button in the Data Manager) would be cumbersome and error prone.
profile picture

Harapa

#15
No problem. :)
profile picture

akar

#16
What is the Data directory location for AlphaVantageStaticProvider ? I downloaded some data AlphaVantage but the C:\Users\Admin\AppData\Roaming\Fidelity Investments\WealthLabDev\1.0.0.0\Data\AlphaVantageStaticProvider is empty.
profile picture

Eugene

#17
Be assured that there's no mystery, it's always here:

X:\Users\Windows username\AppData\Roaming\Fidelity Investments\WealthLabDev\1.0.0.0\Data\AlphaVantageStaticProvider\
profile picture

akar

#18
Thanks. I was looking at the wrong folder. It's fixed.
profile picture

tomphm

#19
For some reasons I couldn't get AlphaVantage to update index symbols like: .SPX, .IXIC, .DJI, etc. I have a premium plan with them so it is not quota limitation problem. All other daily and intraday equity symbols are updating fine. Are they using a different format for the index symbols?
profile picture

Eugene

#20
Yes, like any data vendor. Those symbols are Fidelity specific. Even for Yahoo they're different, and on Morningstar or Reuters website they may be even more different. You should look that up on any vendor's website that you plan to use.
profile picture

tomphm

#21
Thank you Eugene. Will do.
profile picture

Harapa

#22
In my experience AplhaVintage intraday data for most recent trading session becomes available sometime after 10PM EST. But post #9 in another topic suggests(?) that it is available in RT. Please guide on how to configure the provider for this mode.
profile picture

Eugene

#23
I don't see where Cone's posts suggests this. There's nothing to configure in the provider for this mode.
profile picture

Eugene

#24
You're right, intraday data for "today" (even the snapshot quote endpoint!) isn't available during the most recent trading session. This is the case with a free key. I wonder if it's different for Premium subscribers or not?

UPDATE: No it's not. Intraday data becomes available after 8:30PM on Free and Premium plans alike.
profile picture

WayneC

#25
Hi Eugene, I set up two strategies, (a 3 & 5 minute), using AlphaVantage as the data provider. AlphaVantage is setup and enabled in the MM. Updating in the Datamanager looks good, however both strategies stay red in the strategy monitor. The Actions for both show Updat2: MSFT=1820 and Update2: KLAC=1092.

I also get the following message when I try to open the strategies in a strategy window. "On Demand Update Error: Unable to cast object of type Wealthlab.dataProvidersAlphaVantage.AlphavantageStaticSettings to type Wealthlab.dataproviders.yahoo.yahooStaticSettings."

The reference to Yahoo doesn't sound right. Any ideas on how I can get the minute strategies through Alpha vantage to work?
profile picture

Eugene

#26
As we discovered, Alpha Vantage is not suitable for intraday trading in the Strategy Monitor as the data is delayed (becomes available hours after the market close).

It could be due to mixing data by different providers e.g. Yahoo streaming provider enabled, your DataSet is by AlphaVantage. If the S. window error persists please start a new forum topic with more details.
profile picture

tomphm

#27
I just done some back-test simulations comparisons using intraday and daily data from Fidelity and AlphaVantage . The results are quite different and I try to understand what might cause it. I don't think split-adjustment is a contributing factor, since data from AlphaVantage is "split adjustment" using the checked box within the Data Manager.

Data coming from AlphaVantage is limited and is concatenate to that from Fidelity's. The new data from AlphaVantage alone should not cause the significant differences that being observed since it is limited in range. However, it is observed that during data update from AlphaVantage, although the # of new bars being added is small, the # of bars being corrected is much larger, in the range up to few thousands bars (daily) and this occurs for most of the symbols. This says that the bar corrections performed by AlphaVantage covers pretty much all that from Fidelity for a large number of symbols. This could explain the large differences in the simulation results.

This leads to the question, if this is the case, should the data integrity from AlphaVantage is questionable? I have a personal opinion that data from Fidelity's is more accurate than that of AlphaVantage.

Also are there other factors beside that of 'data corrections' done by AlphaVantage that might contribute to the large discrepancy in the simulation results? Thanks.


P.S. I want to add that factors such as data range, code and simulation settings are identical in both simulations. The causes of the differences in the simulation results seems to come from the data.

P.P.S. I do have a Premium plan at AlphaVantage.
profile picture

Eugene

#28
It's expected that data can vary across vendors. Here's a couple of tips:

1. Double click on trades that raise questions in the Trades view to inspect them.
2. Compare charts visually between Fidelity and Alpha Vantage.
3. Check AV's data for validity using the Data Tool.
profile picture

tomphm

#29
Thanks Eugene. What is your opinion with regard to data accuracy between that of AlphaVantage and Fidelity?
profile picture

Eugene

#30
Since I never worked with Fidelity data, I'd rely on the opinion of more experienced users here.
profile picture

KGo

#31
The AlphaVantage provider seems to be repeating symbols from prior datasets in each subsequent dataset. Fidelity did not do this.
Fidelity update log contained:
QUOTE:
Updating 15 Minute data ...
QQQ 105985 bars 7/27/2020 4:00 PM 26 bars added
SPY 105966 bars 7/27/2020 4:00 PM 26 bars added
SVXY 56579 bars 7/27/2020 4:00 PM 26 bars added
VXX 74843 bars 7/27/2020 4:00 PM 26 bars added


Fidelity/AlphaVantage 15 min dataset contains QQQ SPY SVXY VXX VXXB XIV and "Update only symbols that are contained in datasets" is on. XIV and VXXB were in Fidelity quarantine so are not in Fidelity update above. There are symbols in AlphaVantage 15 min files that are correctly not included in the update requests so "Update only symbols that are contained in datasets" is working:
QUOTE:
15 Minute data is being updated...

[0] ^NDX 1625095 bars 8/28/2020 0 bars added
[0] ^VIX 1438104 bars 8/28/2020 0 bars added
[0] MDY 1518826 bars 9/14/2020 0 bars added
[0] MVV 488328 bars 9/14/2020 0 bars added
[0] MZZ 375269 bars 9/14/2020 0 bars added
[0] PSQ 3584 bars 9/15/2020 0 bars added
[0] QID 1324955 bars 9/14/2020 0 bars added
[0] QLD 1291429 bars 9/14/2020 0 bars added
[0] QQQ 107982 bars 9/11/2020 0 bars added
[0] SH 266955 bars 9/11/2020 0 bars added
[0] SPY 107966 bars 9/11/2020 0 bars added
[0] SQQQ 2666 bars 9/14/2020 0 bars added
[0] SRS 162293 bars 9/11/2020 0 bars added
[0] SVXY 58016 bars 9/14/2020 0 bars added
[0] TBF 195172 bars 9/15/2020 0 bars added
[0] TLT 318269 bars 9/15/2020 0 bars added
[0] TQQQ 2667 bars 9/15/2020 0 bars added
[0] VXX 76843 bars 9/14/2020 0 bars added
[0] VXXB 4548 bars 4/29/2019 0 bars added
[0] VXZ 162193 bars 9/15/2020 0 bars added
[0] XIV 46414 bars 2/15/2018 0 bars added
AlphaVantage provider is set to one thread and 1 retry.
profile picture

Eugene

#32
AlphaVantage provider (or for that matter, any other data provider the Fidelity intraday data can be migrated to) does not have any idea of Fidelity quarantine and will not comply with it. That was a feature of the Fidelity static provider.

This aside, are you seeing any issues?
profile picture

KGo

#33
The issue is AlphaVantage provider when doing a all datasets update repeatedly attempts to update symbols that are not in the dataset/scale being updated. The 15 minute dataset example above has 6 symbols in the dataset. The provider attempts to update 21 symbols. The 4 with the lowest bar count are actually from the AV daily dataset. Others are from 1min, and 5 min datasets. There should be 6 symbol attempts not 21.

MDY for example is ONLY in the 1min dataset. Yet it appears as being updated in 1, 5, 15, and 30 min datasets.That's not right.

So it feels like the provider is slower than required and racking up unnecessary API requests against the daily quota. The Fidelity provider did not re-request symbols from already updated datasets. Daily symbols were not requested during the x min updates.


profile picture

Eugene

#34
I will fix this in v2020.10. Thanks for the heads-up.
profile picture

KGo

#35
You're welcome.
profile picture

Eugene

#36
What's new in v2020.10?

* Fix: repeated updates to symbols that are not in the DataSet/scale being updated in "Update All Data" mode update
* Change: .NET 4.6.2+ and WL 6.9+ required

Source: Map of Wealth-Lab data providers
profile picture

tradeoholic

#37
Oh, another question about AlphaVantage: I wanted to download data for symbols, but I found nowhere a symbol list that AlphaVantage uses. Where can I find it ? Or is it the same as the symbol list of Yahoo ?
profile picture

Eugene

#38
QUOTE:
I wanted to download data for symbols, but I found nowhere a symbol list that AlphaVantage uses.

Have you visited the AlphaVantage website and tried asking them?
This website uses cookies to improve your experience. We'll assume you're ok with that, but you can opt-out if you wish (Read more).