QuoteMedia data provider
Author: wl64bit
Creation Date: 11/3/2010 12:10 AM
profile picture

wl64bit

#1
Are there any plans to make a QuoteMedia daily data provider from http://www.quotemedia.com/ ?
profile picture

Eugene

#2
Customer asked, and Wealth-Lab listened: here goes our QuoteMedia daily data provider v.2010.11.

Through QuoteMedia, a quite vast range of free daily data can be downloaded: U.S. and world stocks, indexes, funds, futures, options and more.

Download and install Extension
QuoteMedia provider in the Wealth-Lab Wiki
profile picture

wl64bit

#3
It starts downloading but after some time it gives an error :
"Wealth Lab has stop working".
profile picture

Eugene

#4
* Is the version 2010.11.1? A hotfix was applied.
* If already, which exact symbols?
profile picture

wl64bit

#5
It is WealthLab.DataProviders.QuoteMedia_2010.11.1.
It's a big list with thousands of US symbols.
profile picture

Eugene

#6
It must be due to inefficient handling of network errors. On updating the bunch of symbols for Nth time, QM started returning some unexpected (garbage) results. Updating to build 2010.11.2 should help.
profile picture

wl64bit

#7
It works perfectly now.
Thank you Eugene!
profile picture

wl64bit

#8
Today it gave this error and stopped updating:

Symbol: LXRP, Bars: 2, Last date: 11/9/2010
Symbol: LXRX, Bars: 2, Last date: 11/9/2010
Symbol: LXU, Bars: 2, Last date: 11/9/2010
Error: Object reference not set to an instance of an object.
Update completed (9 min 34.809 sec)
----------------
profile picture

Eugene

#9
Thanks for reporting. This intermittent problem was yet another malformed data issue (returned by QuoteMedia). A huge DataSet would update several times w/o hiccups and then this error would appear out of the blue. Build 2010.11.3 finally takes care of it.
profile picture

wl64bit

#10
Thank you. The new version did not give any error today.
profile picture

Eugene

#11
Good news!
profile picture

redliontrader

#12
Can I get options quotes via the QutoeMedia Provider.. If so How do i enter them since it looks like the format is

SYM YYMMDDC000000 where there is a space between the underlying symbol and the reset of the symbol.


I did get it to work by putting in %20%20%20 between the underlying and the body of the option symbol:

@SPY%20%20%20101120P00120000

for the SPY 120 puts for November for instance.

It would be nice if we could hide that say allow us to put in:

@SPY_20101120P00120000 for the symbol hand have the data provide expand out the spaces %20.

Note.. it must be three space (%20)s not more or less


opps. posted a little too soon.. just found a formating issue above.

The symbol is a fixed 6 character field. So 3 character symbols need %20%20%20, 2 Char symbols need 4 spaces etc..

profile picture

Eugene

#13
Thanks for the heads-up. The reason why this trouble happened was because space is typically considered a separator character rather than a part of a valid symbol name. As it turns out, with QuoteMedia it should be treated differently. A new build of the provider with a bug fix, version 2010.11.4, has just been uploaded. Update by switching to "Other extensions" in the Extension Manager tool.

From now on, any valid option symbol which contains spaces e.g. @SPY 101120C00120000 , should be wrapped in double quotes i.e. "@SPY 101120C00120000". This convention equally applies to the New DataSet Wizard and to modifying a QuoteMedia DataSet (either via the "Symbols" list or by selecting "Add symbols to the DataSet" from the DataSet tree's right-click menu).

Note: when manually adding a new symbol with spaces via the Data Manager's "Symbols" list, you should be careful to wrap all symbols containing space(s), in double quotes, again! It might get tricky so I would recommend to use the right-click option instead.
profile picture

redliontrader

#14
Thanks for jumping on that. Unfortunately there are a couple of things that make it harder to work with now for me.

1. You still have not removed the "strict" formating requirement from QuoteMedia which is a 6 character field requirement after the @ sign so I have to format my symbol string with the extra spaces so "@C 110122C00004000" actually has to be entered as "@C______ 110122C00004000" (I had to re-edit this post as the spaces on the correctly formatted symbol was also stripped, the _ represents spaces). My spreadsheet (google spreadsheets) actually trims the extra spaces so I can not get the symbols now into their proper format. It would be great if the dataprovider could reformat the symbol correctly for QM.

2. I think this might be what you are talking about as warning us away from the data manager, but the symbol list under the data manger strips out the quotes after they are entered thus preventing you effectively from appending more symbols to the list because the current symbols are no longer valid.

thank you..

I don't know what happened, but google chrome posted this a whole bunch of times.. sorry about that. This is the current posting
profile picture

Eugene

#15
1. I didn't promise/intend to remove what you just called "strict" formatting. It's the original symbology of the data vendor, as specified on their website, and should be preserved. I suggest fixing your spreadsheet issues instead.

2. That's what I'm talking about. The DM, by its design, does strip the quotes but if you noticed, it doesn't affect the existing DataSets. They keep working correctly (but every existing symbol with spaces should be taken in double quotes when user starts modifying the "Symbols" list manually).
profile picture

redliontrader

#16
Unfortunately I don't think there is a workaround in google spreadsheets for trimming the extra spaces, i have tried everything I can think of to work around it and now the %20 hack for filling the spaces doesn't work either with the updated quote provider. That had the advantage of both working with the spreadsheet and with the DM since there was no white space.

The strict symbol formating is a side effect of QM's weak PHP implementation, if you read their symbology guide it doesn't mention the fixed field size for the underlying symbol.

I need to ability to append multiple symbols a day and using the right click method is kinda cumbersome.

I will keep thinking.. Thank you for your efforts

-m
profile picture

Eugene

#17
I barely understand the connection between your spreadsheet and the QuoteMedia provider and especially why should it affect the way the provider works.

Furthermore I can't reproduce the issue here as upon entering "@C 110122C00004000" (with 5 spaces eaten by the forum software) in a Google spreadsheet's cell, copying and pasting back the value as text in Notepad produced an exact duplicate.

Re: "the fixed field size for the underlying symbol" - Yes I read their symbology guide which states the opposite:
QUOTE:
VBQ (and three spaces) : Six character option root symbol using any prefix/suffix as defined by the OSI definition; left-justified, space-padded.
profile picture

redliontrader

#18
Thank you Eugene..

You have my work around.. it looks like you understood perfectly.. I need to cut and paste into notepad first.. that works fine for me thanks. brilliant!

I also stand corrected on the Symbology.. i does specifically state 6 spaces.

My only problem now is appending symbols into DM which if you re-apply symbol changes the DM wants to have the legacy symbols wrapped in quotes.. but I will just add the entire list again via the notepad hack.. Wonder why pasting into notepad re-applies the trimmed spaces but pasting into DM doesn't..

The important thing is I am back up and running and grateful.. thanks



profile picture

wl64bit

#19
Second time have this problem:
QM data provider updates the last day but delete the previous day, so the data base is missing one bar. Here is the part of the example from the log:

Symbol: CTQN, Bars: 0
Symbol: CTRN, Bars: 1, Last date: 12/23/2010
Symbol: CTRP, Bars: 1, Last date: 12/23/2010
Symbol: CTS, Bars: 1, Last date: 12/23/2010
Symbol: CTSH, Bars: 1, Last date: 12/23/2010
Symbol: CTSO, Bars: 1, Last date: 12/23/2010
Symbol: CTT, Bars: 2, Last date: 12/23/2010
Symbol: CTTAY, Bars: 0
Symbol: CTTD, Bars: 1, Last date: 12/23/2010
Symbol: CTTY, Bars: 0
Symbol: CTUNF, Bars: 0
Symbol: CTV, Bars: 1, Last date: 12/23/2010
Symbol: CTVIF, Bars: 0
Symbol: CTVWF, Bars: 1, Last date: 12/23/2010

As you may see, only the CTT has been updated for the last 2 bars, thus its data is ok. The rest of the stocks lost the 12/22/2010 day. So, I had to delete the entire database and redownload the whole database.
profile picture

Eugene

#20
It's by design. Zero bars are being reported when QuoteMedia returns some invalid data. Just repeat the update w/o deleting anything. Hope that helps.
profile picture

wl64bit

#21
That does not help, since when I update it second time those symbols considered as updated up to 12/23/2010 and are not updated again.

I would recommend adding an options how many bars to be reloaded while updating the database. It would resolve the problem.
profile picture

Eugene

#22
If they are considered updated up to 12/23/2010, then you already updated them before. The data for 12/23/2010 must exist.

What is the last date for these (problematic) stocks as shown by the Symbol Details window (Data Manager)?

If you change the system date to 12/24 or 12/25 and repeat the date update, does it help?
profile picture

wl64bit

#23
That does not help.
The last day is 12/23/2010. The problem is that when I updated on the 23rd it deleted the 22nd. So, the data is ... 12/20/2010,12/21/2010,12/23/2010. The 12/22/2010 is missing.

profile picture

Eugene

#24
I see what causes this. Will be fixed in v2011.01. Thanks for the heads-up.
profile picture

wl64bit

#25
Got the same problem today (12/27/2010). May be it has to do with holidays...
profile picture

Eugene

#26
No it shouldn't have. With QuoteMedia you get partial bars (incomplete today's bar) so they always have to be removed to ensure data validity. I determined that the issue is caused by removing the last bar on update even when the provider has received 0 bars.
profile picture

Eugene

#27
This provider (and many others) will be updated early next year after WL 6.1 is released (due to some data API changes).
profile picture

Eugene

#28
The QuoteMedia provider has been updated to version 2011.01 on 01/12/2011. Update to Wealth-Lab 6.1 to be able to use it.

Highlights:

* Returns trading session's open price. The new (as of WL 6.1) GetSessionOpen method is fully supported. Check out the QuickRef for more details and a usage example.
* Various fixes: internal fixes, gracefully handles network errors
* Fixed the issue reported by wl64bit: penultimate bar removed when updating even on no new data
profile picture

Eugene

#29
The QuoteMedia provider has been updated to version 2011.11 on 01/12/2011. Update to Wealth-Lab 6.2 to be able to install and use it.

Highlights:

* Added: Market Manager support
* Fixed: needless requests for data updates (should improve the speed of updates in the Strategy Monitor on large DataSets)
profile picture

wl64bit

#30
The QuoteMedia data provider does not download historical data of stocks from Germany but it does from London. Is it possible to fix the problem?
profile picture

Eugene

#31
Problems like this just confirm my supposition expressed here (fyi):

Erratic data from QuoteMedia

It's not the provider fault but a permission issue for the free users. I tweaked something and was able to download German stocks, but in this case nobody's going to make a promise that the fix will stand. It has something to do about the data vendor's intentions. Very well could be that they're starting to curb the free data downloads. I'll give the fix a few more tests, and if it does not affect other downloads, it will be rolled out.
profile picture

wl64bit

#32
Any news about the fixing?
profile picture

Eugene

#33
News after 3 business days?

I sincerely appreciate your understanding that features/fixes need time for testing, not to mention that there are other customers, projects, bugs, tasks, and deadlines to meet.


I'll try to get to it later this month (or early next month at worst). Right now there are other, high priority items I must concentrate on.
profile picture

Eugene

#34
The QuoteMedia provider has been updated to version 2012.05.

Highlights:

* Fix: German stocks data not downloading (reported by wl64bit)
* Fix: uninstalling QuoteMedia provider will keep shared Market Manager library (reported by Gamba)
* Change: user agent upgraded (internal)
* Change: Wealth-Lab 6.3+ required to install this provider
profile picture

cravens

#35
I just installed QuoteMedia data provider in order to get historical options data on SPY.

When I make a new Dataset, I do not see a Data Provider names QuoteMedia. I only see Fidelity, Ascii, MetaStock, MSN, WL4, and Yahoo. Should I see QuoteMedia as a provider?


Rob
profile picture

Eugene

#36
Of course it should, after restarting WL6. Otherwise see the FAQ:

Extension installed in Wealth-Lab Developer doesn't show up after restarting application.
profile picture

cravens

#37

I installed the extension and the .dll is in the Wealth-lab main folder. I restarted. I am using WL 6.3

Still not seeing QuoteMedia as Data Provider.

Sounds like this may be the problem. Please advise.
From FAQ - "targeted the right framework version. To be recognized by Wealth-Lab 6.3 and earlier versions, libraries have to be compiled against .NET 2.0 - 3.5. By default, Visual Studio 2010 targets .NET 4.0 (incompatible with WL 6.3) - this can be changed in your project's properties, and it should be recompiled."

Rob
profile picture

Eugene

#38
I think I know what's happening. Somehow, Market Manager - a required component of the provider - is missing from your installation. To get it back, reinstall from the Extensions section of the website any provider that is using it. I suggest ASCII or Google, for example. Then restart WLP and everything should be fine. Good luck.
profile picture

cravens

#39
Thank you. ASCII worked.

So now I enter a new symbol list with the symbol in double quotes like so:

"@SPY 121020C00144000" to get the oct 20, 2012 144 call and I specify the data source as QuoteMedia. I also add this to the Market Manager .

When I update data I get:
Updating Provider QuoteMedia ...
Updating daily data from QuoteMedia...
Up-to-date symbols: 0, Update required for: 0, New symbols: 1
Updating new symbols...
Symbol: @SPY 121020C00144000, Bars: 0

Provider update complete for QuoteMedia
----------------

Update completed (0.531 sec)

There is no data associated with the symbol to plot.

Any thoughts?

Rob
profile picture

Eugene

#40
Glad to help.

Re: @SPY... option symbol. Either there is no downloadable data for that symbol, or the symbol is not correct.
profile picture

cravens

#41


I assumed that I had a bad symbol so I went to www.quotemedia.com and got a current symbol from their option chain.
The symbol is @SPY 121020C00143000
When I go to the Quotemedia section to enter the symbol and get a quote it does indeed give a quote and a chart as I expected.

I then entered this symbol (in double quotes) into Wealth-lab symbol list and then used market manager to 'copy from symbol list'.
I used update data to try to get data into WL and still no data.

Any thoughts?

profile picture

Eugene

#42
Entering into Market Manager isn't really required in this case.

At the risk of repeating: there is either "symbol incorrect" - it is correct - then it's "no downloadable data for that symbol". This is a free data source, and stuff like that happens: a symbol's data exists on the web though it's inaccessible for their API calls. We will have to live with this.
profile picture

Eugene

#43
But wait, there's more. It's neither #1 nor #2 but it is #3: the symbol's data exists but either you have typed it incorrectly in Wealth-Lab (hint: use copy/paste instead of typing in) and/or the website has eaten a portion of it.

I was curious enough to enter your symbol @SPY 121020C00144000 into their symbol lookup page and of course the symbol is invalid and couldn't be charted.

The correct symbol name coming from Quotemedia's website is "@SPY___121020C00144000". Replace the underscores with spaces - the forum does not seem to like multiple spaces. Do you notice the three spaces?

Hope that helps.
profile picture

cravens

#44

ok. That made a big difference. I can now go to quotemedia's site and get any option data I want.

I also deleted and then re-made my dataset in WL and now all is well.

Thank you..

I need to go backtest my strategy....

Rob


profile picture

wl64bit

#45
It looks like QuoteMedia stopped working. Could you please check if it works for you?
profile picture

Eugene

#46
Thanks for the heads-up. No worries: their data format has changed slightly. Please stand by while our webmaster re-implements extension uploads for the new website (no ETA currently but it's a priority). We'll fix the provider and upload.
profile picture

wl64bit

#47
Quotemedia allows downloading data any time during the day, so is it possible to allow updating the data after 16:30 EST?
profile picture

Eugene

#48
The provider uses its own medium for downloading the data. It's not based on the "Historical quotes" section of the website. There's no imposed restriction in the provider, only the data vendor decides when to make the data available.
profile picture

Eugene

#49
The QuoteMedia provider has been updated to version 2013.05.

It's a mandatory service release (previous versions stopped due to sudden data format change).

Also, added named DataSeries "AdjClose" with adjusted close.
profile picture

Eugene

#50
kabjian asked:

Hello,
I am having trouble loading option chains using QuoteMedia extension.

I reviewed the previous forum posts and wiki entry on this topic. I enclosed my symbols in double quotes for example: "@SPY 101120C00120000" and "@SPY 150417C00130000" and "@XLF 150417C00024500". I have tested these and other chains individually, wrapped them all in double quotes and I cannot seem to return the time series. QuoteMedia does work for the underlying symbols but not the chain.

Any help would be greatly I appreciated. Thanks.
profile picture

Eugene

#51
Hello,

1. As can be seen, no problem with the first symbol "@SPY 101120C00120000" at all:



2. The other two symbols are either not recognized by the feed or the server does not return data for whatever reason. It's not the extension's fault.

UPDATE:

Indeed, you've inserted an unsuitable number of spaces between the @SPY or @XLF and the following sequence. It is variable. The correct symbols may be:

"@SPY 150417C00130000" (3 spaces)
"@XLF 150417C00024500" (2 spaces)

See post #17 above for explanation.

P.S. I added this note to the Wiki:

QUOTE:
For options symbols, the number of spaces between the two parts of the ticker can vary! Make sure to enter the symbol exactly as found on the QuoteMedia site, preserving all the spaces.
profile picture

kabjian

#52
Thank you for response the Eugene.

I see your time series above for "@SPY 101120C00120000". I'm curious why an option contract that expired in 2010 is showing data through April of 2015.


From the QuoteMedia site at http://www.quotemedia.com/quotetools/symbolHelp/SymbolHelp_US_Version_Default.html

Options symbols are broken down as follows:
Example: 2011 Jan 22 20.00 Call for Microsoft Corp.
Symbol : @MSFT 110122C00020000
@ : All Options begin with the " @ " character.
MSFT (and two spaces) : Six character option root symbol using any prefix/suffix as defined by the OSI definition; left-justified, space-padded. In the case where the option root symbol is three characters, it will be followed by three spaces.
11 : two least significant digits of the Contract Date year
01 : two-digit Contract Date month
22 : two-digit Contract Date day of month
C : "C" for CALL or "P" for PUT
profile picture

Eugene

#53
QUOTE:
I'm curious why an option contract that expired in 2010 is showing data through April of 2015.

No idea but I won't put too much trust in this provider's data because it's driven by a demo API key and is known for data errors. I guess it shouldn't happen with a paid API key but we haven't had a chance to test that. I'd suggest to double check everything.

P.S. As our Forum's software has been eating spaces, the two tickers in my post #63 should look differently:

"@SPY 150417C00130000" (3 spaces)
"@XLF 150417C00024500" (2 spaces)
profile picture

richard1000

#54
Problems in QuoteMedia provider:

Updating DataSet QuoteMedia Sectors ...
Up-to-date symbols: 0, Update required for: 68, New symbols: 0
Error setting min/max download threads
Error setting min/max download threads
Update completed (0.218 sec)
profile picture

Eugene

#55
This bug occurs under very rare conditions. I'll upload an updated extension (ETA end of month). Thank you for the heads-up.
profile picture

Eugene

#56
What's new in QuoteMedia provider's recent version 2016.05?

* Fix: "Error setting min/max download threads" message prevents data update
* Fix: removed artificial limit (.NET default) in New DataSet Wizard preventing from creating DataSets which contain more than ~6500 symbols (32767 chars)
* Change: .NET 4.5 and Wealth-Lab 6.8+ are required to install/update extension
* Change: (internal) updated browser agent version
* Change: (internal) increased thread download timeout to prevent failed requests
* Change: (internal) multithreaded download routine rewritten. Should improve work on modern multi-core CPUs (e.g. servers)
profile picture

Eugene

#57
What's new in version 2017.10:

* New: support for automatic bar data corrections
profile picture

Eugene

#58
What's new in 2017.12?

* Fixed: Quotemedia provider company names broken
profile picture

marcomaier

#59
Hello,
if I want to update data with my QuoteMedia-Provider it works for my DataSet consisting of 100 symbols. It does not work for my DataSet that consists of 300 symbols - see attachment. My version of QuoteMedia is 2017.12.1.
Do you have any ideas to solve the problem?

profile picture

Eugene

#60
Hello,

What is the symbol(s) that fails to update? If you want me to have a look at it please break down your DataSet in chunks and run updates until you find the misbehaving ticker.
profile picture

marcomaier

#61
Hello Eugene,

if I update 100 symbols DataSet it is working. If I update the DataSet consisting of 500 symbols (that the 100 symbols contains) it is not working - see attachments.
profile picture

Eugene

#62
Marco,

Knowing that some 500-symbol DataSet breaks with an error during update for you is useless for troubleshooting. I already knew it from your last response last year and this knowledge won't help neither you nor me.

At the risk of repeating, if you want me to have a look at the problem please break down your DataSet in chunks and run updates on them until you find the "misbehaving" ticker(s) that throws the error. Or at least attach a text file containing the DataSet's symbols. Then I might have something to start with and hopefully be able to reproduce the issue (unless it's caused in a corrupt *.WL data file on your end - which would be a good clue).

Thanks.
profile picture

marcomaier

#63
Hello,

here is the DataSet ...

Thanks for checking ...
profile picture

Eugene

#64
I did a historical data backfill and like I suspected there's no error:

QUOTE:
Updating DataSet _test QuoteMedia_ ...

Up-to-date symbols: 0, Update required for: 0, New symbols: 500
Updating new symbols...
Symbol: AAL, Bars: 3090, Last date: 04.01.2018
Symbol: A, Bars: 4554, Last date: 04.01.2018
...
Symbol: WAT, Bars: 5464, Last date: 04.01.2018
Symbol: XEL, Bars: 8030, Last date: 04.01.2018
Update completed (6 min 32.41 sec)
----------------


I've even truncated the last N bars and ran a quick update w/o an issue so far. I'm pretty sure everything works as designed but some of your *.WL files may be corrupt. To verify:

1. With hidden files visible, navigate here:

c:\Users\Windows account\AppData\Roaming\Fidelity Investments\WealthLabDev\1.0.0.0\Data\QuoteMediaProvider

2. Delete the "Daily" subfolder

3. Update the DataSet from scratch

4. If problem is reproducible, break the failing DataSet into smaller and smaller chunks and update them until you find the conflicting symbol.

The ball is in your court now!
profile picture

marcomaier

#65
Ah, ok, the symbol "SBAC" causes the error: If I exclude this it works

But I dont't know the reason; in other DataSets it works
profile picture

Eugene

#66
Odd enough. I'd recommend to delete the SBAC.WL file and get its data from scratch.

For my part, I'll commit a potential fix for this type of issue in v2018.02 of the provider. Sorry for the inconvenience.
profile picture

marcomaier

#67
No problem, thanks ...

Now, after deleting SBAC.WL and updating again with SBAC ist works
profile picture

Eugene

#68
Quotemedia provider updated to 2018.02

Fix: potential fix for "un-representable DateTime" error during data update
profile picture

Eugene

#69
What's new in v2018.04:

* Fix: company names broken by Google Finance website redesign
profile picture

Eugene

#70
What's new in v2018.12:

* Fix: more robust fix for potential un-representable date/time error on data update
profile picture

Panache

#71
Is there a reason provider downloads unadjusted QuoteMedia data, as opposed to "Adjusted: Splits Only", which is an option on their website? Since there is no split adjustment for QuoteMedia, this results in some nasty price gaps, ie. AVP between 12/29/1989 and 1/2/1990.
profile picture

Eugene

#72
I just took a look into it. The provider uses a historical data endpoint different than the one that powers their website's Price History. While it does contain both unadjusted and adjusted close series, QM's legacy endpoint is not capable of returning the split coefficient (unlike Yahoo). Without it provider cannot tell a huge gap from a real split. The rest (OHL) are also unadjusted so mixing the adjusted Close with unadjusted OHL wouldn't make any sense.

There's no plan to switch to the newer endpoint (which isn't ideal) so let it remain as is for now. By the way, an undocumented feature is that the provider returns the adjusted close as NamedDataSeries. Just added this to the Wiki: QuoteMedia daily data provider. Currently it works for on demand data update only and is broken for Update DataSet / Update All Data but I'll fix that in next month's update.
profile picture

Panache

#73
QUOTE:
The rest (OHL) are also unadjusted so mixing the adjusted Close with unadjusted OHL wouldn't make any sense.

I'm sorry. You are correct that the QM data (adjusted or unadjusted) isn't any good before 1/2/1990. However, the adjusted OHLC data appears to still be accurate as of 1/2/90, so I think at least fetching that would be better than the completely unadjusted data (with prices in the 30's) currently being returned by the provider.
profile picture

Eugene

#74
That's what the endpoint returns, there is no adjusted OHL to save:
CODE:
Please log in to see this code.
profile picture

Eugene

#75
What's new in v2020.02:

* Fix: undocumented Named DataSeries (AdjClose) is empty except for on demand data update
* Change: targets .NET 4.6.2+
profile picture

pichu_1972

#76
Hello, I think it is not downloading data from quotemedia

Updating DataSet Divisa QuoteMedia ...
Up-to-date symbols: 0, Update required for: 15, New symbols: 0
Symbol: $EURHKD, Bars: 0
Symbol: $EURINR, Bars: 0
Symbol: $EURCHF, Bars: 0
Symbol: $EURDKK, Bars: 0
Symbol: $EURGBP, Bars: 0
Symbol: $EURAUD, Bars: 0
Symbol: $EURCAD, Bars: 0
Symbol: $EURCZK, Bars: 0
Symbol: $EURHUF, Bars: 0
Symbol: $EURNOK, Bars: 0
Symbol: $EURJPY, Bars: 0
Symbol: $EURMXN, Bars: 0
Symbol: $EURPLN, Bars: 0
Symbol: $EURSEK, Bars: 0
Symbol: $EURUSD, Bars: 0
Update completed (0.832 sec)
----------------
profile picture

Eugene

#77
Supposedly they have decommissioned the endpoint which our provider's been using. Quotemedia is broken (not sure if beyond repair yet).

But there are alternative free data providers with access to Forex data like Forexite, Dukascopy, Finam, MFD, AlphaVantage. (Yahoo's forex data is not adequate and is not recommended.) Please don't discuss them in this thread, though.


UPDATE: our QuoteMedia provider will be restored soon and will become better than ever.
profile picture

Eugene

#78
What's new in v2020.03:

* Fix: provider completely restored after QM decommissioned its (legacy) API endpoint
* Change: data feed returns the genuine Bars.SecurityName as reported by QM
* Change (breaking): OHLC data is now split- and dividend-adjusted
* Change (breaking): removed UnadjClose named series
* New: added unadjusted data as Named DataSeries: UnadjustedOpen, UnadjustedHigh, UnadjustedLow, UnadjustedClose
* New: added fundamental provider for splits and dividends ("Split (QuoteMedia)" and "Dividend (QuoteMedia)")
* New: possible to switch between split- and dividend-adjusted data and raw data (see new checkbox in Data Manager > QuoteMedia tab)
* New: added streaming data provider (EOD, delayed)
profile picture

Panache

#79
Thanks Eugene!

On my wish list, it would be nice if the provider didn't terminate (with the not particularly helpful "Error: One or more errors occurred.") if it encounters a symbol which is not valid in QuoteMedia, like "BRK-B" instead of "BRK.B".
profile picture

Eugene

#80
Will be fixed in v2020.04. Thanks and glad to help.
profile picture

Panache

#81
I hate to be the bearer of bad news, but I think the new provider has a bug in the way it handles volume. Here is a summary of the raw and split adjusted price data for AAPL on 12/30/1994:

FidelityStaticProvider\Daily\A\AAPL.WL
12/30/1994 $1.4062 $1.4241 $1.3839 $1.3929 18,272,818 - Raw data is split adjusted

QuoteMediaProvider\Daily\A\AAPL.WL
12/30/1994 $1.1181 $1.1323 $1.1002 $1.1073 73,259,222 - Raw price data is split adjusted. VOLUME IS WAY OFF!!!!
QuoteMedia Splits
6/16/1987 12:00:00 AM 0.5
6/21/2000 12:00:00 AM 0.5
2/28/2005 12:00:00 AM 0.5
6/9/2014 12:00:00 AM 0.142857

TiingoStaticProvider\Daily\A\AAPL.WL
12/30/1994 $39.3800 $39.8800 $38.7500 $39.0000 652,600
Tiingo Splits
6/9/2014 12:00:00 AM 7.000007
2/28/2005 12:00:00 AM 2
6/21/2000 12:00:00 AM 2
6/16/1987 12:00:00 AM 2
12/30/1994 $1.4064 $1.4243 $1.3839 $1.3929 18,272,818 - Split adjusted raw price and volume data

ZZWealthDataStaticProvider\Daily\A\AAPL.WL
12/30/1994 $1.4063 $1.4241 $1.3839 $1.3929 18,422,628 - Raw data is split adjusted

YahooStaticProvider\Daily\A\AAPL.WL
12/30/1994 $39.3750 $39.8750 $38.7500 $39.0000 18,272,800
Yahoo Splits
6/9/2014 12:00:00 AM 7
2/28/2005 12:00:00 AM 2
6/21/2000 12:00:00 AM 2
6/16/1987 12:00:00 AM 2
12/30/1994 $1.4063 $1.4241 $1.3839 $1.3929 18,272,800 - Split adjusted raw price data

Wealth-Data's volume is always a little different that the other providers, so we'll ignore that, although it is still in same order of magnitude. Fidelity, Tiingo and Yahoo all have the same volume within 100 shares (Yahoo's volume is rounded to the nearest 100 shares). However, as you can see, the QuoteMedia volume is way off. I think the problem is that the QuoteMedia split values are 1/x compared to the other providers and the provider uses the same formula for adjusting the volume, ie. raw volume divided by the cumulative split value.

I realize the price data is off also, but that appears to be an error on QuoteMedia's part. (See the attached screenshot).
profile picture

Eugene

#82
Thanks for the heads-up. Will be also fixed in v2020.04.

Note that you will have to reload your accumulated QuoteMedia data for the fix to apply. Also, QuoteMedia provider requires reloading accumulated QuoteMedia data after a split, too (but I guess you're used to it).

Unlike certain providers (especially Yahoo) where split/dividend adjustment is hard to maintain, I take a simple approach in the QuoteMedia provider.
profile picture

Panache

#83
Thank you Eugene.

QUOTE:
Note that you will have to reload your accumulated QuoteMedia data for the fix to apply.

Version 2020.03 obviously created different raw data. Will accumulated data downloaded with version v2020.02 or earlier be usable by v2020.04 (until the next stock split)?
profile picture

Eugene

#84
QUOTE:
Will accumulated data downloaded with version v2020.02 or earlier be usable by v2020.04 (until the next stock split)?

Yes.

QUOTE:
Version 2020.03 obviously created different raw data.

Right, because feed delivers different data now.
profile picture

Eugene

#85
What's new in v2020.04:

* Fix: gracefully handles invalid symbols
* Fix: volume is incorrect when split/dividend adjustment is disabled

You must reload any accumulated QuoteMedia data you may have for the volume fix to apply!

Remember to reload the accumulated QuoteMedia data for a symbol after split! Unlike Yahoo (Tiingo, AlphaVantage) this is not handled by the provider automatically.
profile picture

Panache

#86
Thanks Eugene. I'm watching the QuoteMedia provider download data as I type, and it seems to be working very well.


I spoke too soon. When I started looking at the data this morning, I realized that it exited last night at FCX. I ran it again this morning, and it downloaded data for another 25 symbols or so before exiting with "Error: One or more errors occurred." Attached is the LastUpdateLog.txt if it's helpful.

Since I know you're going to say I have a lot of symbols in my DataSet, I created a new DataSet for just the symbols beginning with F and used "Update DataSet (Pricing)" on that DataSet. It downloaded data for another 15 symbols or so before again exiting. Also attached is the LastUpdateLog.txt for that update.
profile picture

Eugene

#87
I've updated these 86 symbols without a hitch. Looks like a temporary network/server issue took place during your update and provider faced some unexpected input. Stay healthy - it's more important now!
profile picture

Panache

#88
Thanks. I'm still having the same issue. If it's not too much trouble, could you try to download my F DataSet which is attached? If you can, then obviously, the issue is local to me.
profile picture

Eugene

#89
Given the complete DataSet now this error was easy enough to catch. Update the extension to v2020.04.1 and it should run update to the end.
profile picture

Panache

#90
You got it. v2020.04.1 went through my entire DataSet with no problems. Thank you and stay safe.
profile picture

Panache

#91
I really feel bad having to say this, but the new provider is picking up the Adjusted: Splits and Dividends data, rather than the Unadjusted data. As a result, the price data is very different from what the other providers are returning. Attached is a print for AAPL before and after the most recent split.


I also realized I was still applying the Fidelity split factor for QuoteMedia, so my split numbers will look very different. Attached is a corrected print for AAPL and a screenshot from QuoteMedia showing their Unadjusted data
profile picture

Eugene

#92
This should make you feel good:

QuoteMedia provider user guide > Splits and Dividends
profile picture

Panache

#93
Thank you. Maybe I'm unusual, but I use split, but not dividend, adjusted data. It should only take a couple of extra lines of code to add the dividends back in. I'll add that to my "to do" list.

Since I only use raw QuoteMedia data, adding a checkboxes to adjust for dividends and splits separately like you have for Tiingo and Yahoo is not something that would help me, but others may find it useful.


OK. I'm not as smart as I thought.

The raw data Close for APPL on 1/31/2020 is 308.7771. I would have thought that if I added .77 (the Dividend (QuoteMedia).Value for the dividend on 2/7/2020, I would get the unadjusted Close of 309.51 (which is what is shown in a Wealth-Lab chart for QuoteMedia AAPL, with Split and dividend adjustment disabled). It is close, but off by 0.0371. Please tell me what I'm doing wrong.



profile picture

Eugene

#94
Provider returns both adjusted or unadjusted data as delivered by QM. With regard to split/dividend adjustment it does no processing of the data whatsoever. Unlike Yahoo or Tiingo providers, this difference also explains why one has to reload data after split/dividend.

Simply put, the data comes from QM as is and the provider only reflects the raw values. Hope this helps.
profile picture

Panache

#95
QUOTE:
With regard to split/dividend adjustment it does no processing of the data whatsoever.
Now I'm really confused. When I run this code, I get very different results, which are attached.

CODE:
Please log in to see this code.
The first run is with the DataManager>QuoteMedia>Split and dividend adjustment radio button Enable activated and the second is with Disable activated. As you can see, the data is different depending on whether it is accessed via Bars/GetExternalSymbol() or LoadFromFile(). (I don't need to know why the bar numbers are different.)

You may recall that a few years ago we discussed the problem that GetExternalSymbol() may return data from a different data provider if the provider does not contain data for that symbol. https://www.wealth-lab.com/Forum/Posts/GetExternalSymbol-DataSet-32124 Since then, I've always used LoadFromFile() which guarantees the provider.

QUOTE:
Provider returns both adjusted or unadjusted data
How can I access the unadjusted data using LoadFromFile()?
profile picture

Eugene

#96
I don't recall said discussion but as per the QuickRef, you guarantee the provider by specifying the dataSetName parameter in GetExternalSymbol. Accessing the unadjusted data should only be approached as documented in the online user guide > NamedSeries. Hope this clarifies.
profile picture

Panache

#97
Thanks. Using FindNamedSeries should do what I need it to.

QUOTE:
specifying the dataSetName parameter in GetExternalSymbol
. Just fyi, if you specify the dataSetName, GetExternalSymbol returns the unadjusted data, regardless of the setting in Data Manager.
profile picture

Panache

#98
I'm sorry to keep checking the QuoteMedia provider, but I find QuoteMedia to be a very useful source for historical data that isn't available elsewhere.

QUOTE:
Provider returns both adjusted or unadjusted data as delivered by QM.
Unfortunately, that not what I'm seeing when I look at the data. The "Adjusted: Splits and Dividends" Historical Data for ABMD on 2000-09-29 from the QuoteMedia website shows a volume of 119.10k (Website.jpg). When I set Data Manager > QuoteMedia > Split and dividend adjustment to "Enable", the volume is 238,200 (Wealth-Lab.jpg), which is what is contained in the ABMD.WL file.

QUOTE:
With regard to split/dividend adjustment it does no processing of the data whatsoever.
There was a 2:1 stock split the next day, and it looks to me like the provider multiplied the QuoteMedia volume by 2, because if I set the Split and dividend adjustment to "Disable", I get the 119,100 volume which is what QuoteMedia reports for their "Unadjusted" Historical Data.

I can work around whatever the provider is doing, but it would be very helpful for me to know what is raw data and when the provider has adjusted the data. Is there a way to get the unadjusted volume?

(Of course, the split adjusted and non-split adjusted volume should not be the same. That is one of the problems with the QuoteMedia data; sometimes you have to adjust the volume for splits and sometimes not.)
profile picture

Eugene

#99
I looked into it and there's no error.

The unadjusted volume figure on 09/29/2000 i.e. 238,200 in the raw data (sorry I will not disclose how to arrive at it) predictably matches WL's number as provider just reflects what it collects from the feed. With "Unadjusted" option selected on the website, the volume displayed there for 09/29/2000 does not align with their own raw numbers. This looks like a website bug to me.

Unlike Yahoo/Tiingo, the Quotemedia provider does not massage the data. Like I already stressed, this also requires its users to reload the data after each split. There is no multiplication or division taking place.

Provider is working by design. No modification is necessary.
profile picture

wl64bit

#100
Quote Media Provider has an issue. It downloads the data with the last date of 9/2/2020 but shows the data on the chart only for new Symbols. For old Symbols that already existed in the dataset, the newly added bars are not shown on the chart since the O-H-L-C data is always 0.

After Reloading Chart History from the chart itself, the newly added bar(s) appear.
profile picture

Eugene

#101
Works fine here (2020.04.1):

Updating DataSet 123 ...
Up-to-date symbols: 0, Update required for: 0, New symbols: 3
Updating new symbols...
Symbol: TSLA, Bars: 2564, Last date: 02.09.2020
Symbol: QQQ, Bars: 5398, Last date: 02.09.2020
Symbol: AAPL, Bars: 9073, Last date: 02.09.2020
Update completed (11.12 sec)
----------------

After truncating all bars after 8/1/2020 using the Data Tool to emulate your issue:

Updating DataSet 123 ...
Up-to-date symbols: 0, Update required for: 3, New symbols: 0
Symbol: TSLA, Bars: 25, Last date: 02.09.2020
Symbol: QQQ, Bars: 25, Last date: 02.09.2020
Symbol: AAPL, Bars: 25, Last date: 02.09.2020
Update completed (2.621 sec)
----------------

Cannot reproduce.

If you would like me to look into it please consider coming up with a detailed, reproducible test case and logs.

P.S. Just FYI. We employ a watchdog that alerts if any data scraper in the wild (all data providers, their Classification Groups, C.Components, public Strategies that downloads data on-the-fly, countless other utility scripts etc.) goes down or starts returning unexpected results. We're watching it closely and QuoteMedia has been working just fine.
profile picture

wl64bit

#102
I think this might help. Trying to reproduce it during the trading day. I have deleted the QuoteMediaProvider and QuoteMediaFundamentalProvider folders. Created a dataset with just AAPL.

Case 1 (working CORRECTLY)

Configuration:
Split and dividend adjustment: Enable
Always return data with partial bar: checked

after updating the dataset, here are the last several bars of the chart copied to the clipboard:

8/28/2020 126.0125 126.4425 124.5775 124.8075 187629916
8/31/2020 127.58 131 126 129.04 225702688
9/1/2020 132.76 134.8 130.53 134.18 152470142
9/2/2020 137.59 137.98 127 131.4 200118991
9/3/2020 126.91 128.84 126.27 127.1 388156

Case 2 (working INCORRECTLY)

Configuration:
Split and dividend adjustment: Disable
Always return data with partial bar: checked

after updating the dataset, here are the last several bars of the chart copied to the clipboard:

8/28/2020 504.05 505.77 498.31 499.23 46907479
8/31/2020 127.58 131 126 129.04 225702688
9/1/2020 132.76 134.8 130.53 134.18 152470142
9/2/2020 137.59 137.98 127 131.4 200118991
9/3/2020 0 0 0 0 0

profile picture

Eugene

#103
In any case, you should reload a symbol's QM data following a split or dividend. This is documented in the online user guide. AAPL is a good example as a stock with a recent split. Remember, this provider is not like Yahoo where this is smoothly handled under the hood.

I suggest reopening the chart window after making changes to QM's "Split and dividend adjustment" option.
This website uses cookies to improve your experience. We'll assume you're ok with that, but you can opt-out if you wish (Read more).