Database fundamental provider request
Author: pfelix
Creation Date: 2/28/2011 11:05 AM
profile picture

pfelix

#1
I've been reading through the many fundamental data API requests and understand that the API is not yet documented. Are there any plans to add a database source for fundamental data? Such as WealthLab.DataProviders.Databases.Fundamental.dll. I'm mostly interested in getting proprietary data into WL for Neuro-Lab use along side YCharts data. This can be accomplished in a workaround fashion with the current database provider by importing the data as if it were a static quote, but I'd rather get my non-symbol specific data into the appropriate part of the application.

Thanks
profile picture

Eugene

#2
What particular "database source for fundamental data" do you mean? Please clarify by providing as much details as possible.
profile picture

pfelix

#3
I am looking for something very similar to the existing database provider for static & streaming data except that it would import fundamental data not OHLCV quotes. This data would be available from the fundamental data list in a separate folder. Ideally there would be an option to define if the data is symbol specific. Nearly all of the data I want to import is environmental (economy, weather, political, etc...). So, I would envision the provider working like the YCharts Economic & Fundamental provider does today with the exception that there would be a ui for defining a connection & mapping the fields.

Step 1 - Define Connection to db (existing providers are fine - Oracle, SQL, ODBC, OLEDB) & Table / View to import
Step 2 - Map fields (would not have ohlcv; just current value). Inputs would include...
o Name - Caption for fundamental such as "LaborRelatedDataItems"
o Symbol - If the data is symbol specific, then define a symbol field. Otherwise the data is not symbols specific (same data for all symbols).
o Value - The item value
o Description - Optionally provide a description (shown on bottom of fundamental window; context sensitive)
o DateTime - Timestamp; correlates with bar when synchronized is set to true
Step 3 - Create fundamental dataset; each unique "Name" becomes a fundamental item
Step 4 - View fundamenta/economic data in the fundamental data list
o Items can be plotted
o Items can be referenced with WealthScript via GetExternalSymbol or SetContext methods

Basically, what I'm looking for is the current database static provider except with a couple changes to the inputs (above fields) and the outputs are displayed as fundamental items instead of ohlcv quotes.

Does that make sense?

profile picture

Eugene

#4
Thanks for the thorough description. Too many don'ts:

1. There is no such thing as a "Fundamental DataSet".
2. It's not possible to create a static DataSet returning FundamentalDataItems.
3. Static data providers can not serve non-symbol specific data. Although one could be made to return anything in request, symbol name is still the unique key. Only fundamental data can be non-symbol.
4. GetExternalSymbol and SetContext does not apply to fundamental data items.

Given the above, it makes sense to simplify and import your economy, weather, political etc. items either as DataSeries (e.g. FillSeriesFromFile, it works with files so it's rather an illustration of the concept) or as symbol-specific static data.

P.S. Please see my reply to your ticket 20646.
profile picture

pfelix

#5
Couple follow up questions...

- What do you call the data loaded into the "Fundamental" feature of Wealth Lab pro?
- If fundamental data cannot be static, does that mean it needs to be streaming? Please explain?
- My mistake on GetExternalSymbol & SetContext reference. I should have referred to FundamentalDataSeries.

I understand the suggestion to import data as a DataSeries, and I'm able to accomplish my goal by doing this. However, my data fits much better into the "fundamental" part of the application. Please let me know if there is any way currently to get my data which is stored in a SQL Server db into the available list of fundamental items?

Here is a little more information about my data. I am pulling data from many different sources and storing it at the most granular level possible. For example; with YCharts (I subscribed) we get gdp, gdp growth, and a few other gdp related items. That is too high level for the type of mining that I'm doing. My data has all of the details line items that make up gdp such as durable, non durable, etc... Mathematical transformations such as growth are simple to create, but they are limited in their use when trying to train a model that will sufficiently generalize the impact of each independent variable. Hypothetical example; gdp may not correlate at all with my output (dependent variable) where exported goods does. I have thousands (and growing) of such detail time series that I would like to attempt to utilize in the process of designing profitable strategies.

I also am open to sharing my data so that others with similar interest can benefit & contribute. If the solution is to import the data using the existing database provider, then I will live with that. But, I'm pretty sure that traders who want to do serious quantitative analysis are going to need the ability to do similar imports. Please consider this option of making a database extension that will import fundamental data (symbol specific or not). Alternatively, please tell me how YCharts supplies data to the YChart extension. I am open to replicating this type of data feed so that you can reuse the same code with few tweaks for identifying the source location.

Thank you for your time. I am already a big fan of WLP!
Paul

PS: I will post the required reply to ticket 20646 as soon as the market opens.

profile picture

Cone

#6
QUOTE:
- What do you call the data loaded into the "Fundamental" feature of Wealth Lab pro?
A DatSet consists of symbols that have price data, and they may also have associated fundamental data.

QUOTE:
- If fundamental data cannot be static, does that mean it needs to be streaming? Please explain?
Fundamental data is from the 10-Q, plus other miscellaneous info like dividend and splits. It is static. See the first answer about what a "DataSet" is.

Just jumped in for that, will let Eugene handle the rest.
profile picture

Eugene

#7
QUOTE:
- If fundamental data cannot be static, does that mean it needs to be streaming? Please explain?

Static data providers return data on a per symbol basis and inherit from StaticDataProvider interface. Fundamental data providers are derived from FundamentalDataProvider and can return either symbol-based or non-symbol data. The both interfaces are dealing with their own kind of data, so I simply meant to tell that there's difference between fundamental data providers and static data providers as we know them.
QUOTE:
- My mistake on GetExternalSymbol & SetContext reference. I should have referred to FundamentalDataSeries.

Good clarification Paul, the picture's getting clearer. You were right from the start, a "Database fundamental data provider" seems like the way to go. Of course, this was never suggested before when it was designed so this never was considered. The highly specific data provider you're proposing is undoubtedly interesting but at the same time is a glaring illustration of a single-user solution for which, unfortunately, we don't have the time for. It would be optimal if you could set the DB provider to import the economic data items from the DB tables as regular stock symbols and consume them with GetExternalSymbol/GetExternalSeries.
profile picture

pfelix

#8
Understood. Thank you for taking the time to consider this feature request. I'll use the existing database provider as suggested. I remain very interested in the Fundamental Provider API and welcome the opportunity to contribute to the community providers by developing an extension that is capable of importing fundamental data.

Thanks,
Paul
profile picture

abegy

#9
I would like just to say that I'm interesting also for this feature. This will be a good help to design more powerfull trading system.
profile picture

Eugene

#10
It's not clear to me which "this feature" do you mean exactly, considering that the topic starter has a tailored, highly custom database solution that includes a number of unnamed data sources. Could you please clarify?
profile picture

abegy

#11
At this time, I'm thinking to switch from my current data provider (Reuters Datalink) to a new one (QuoteCenter).
With QuoteCenter, there are more than 700 fundamentals data. With this, I will be able to build trading system similar to "Joel Greenblatt" http://www.magicformulainvesting.com/about_book.html

As I'm from France, your current fundamentals data interface (Yahoo, MSN...) cannot be used for me because symbol code are not the same with my data provider. Moreover, I cannot use MSN or Yahoo as my data provider because the quality is very poor for none american market.

For this reason, It would be great if you can give documentations on how to develop Fundamentals interface with Wealth-Lab or add functionnality to Database Provider to use Fundamentals data.
profile picture

Eugene

#12
Unless I'm mistaken, your request seemingly has nothing to do with the Database provider: the topic starter already has a database solution he'd like to have imported by the DB provider, whereas my impression about QuoteCenter is that they provide data in MetaStock format. A different league.

It would be great if the WL6 Fundamental API was disclosed but I doubt that such documentation exists. On the other hand, if QuoteCenter actually provides the data in MetaStock format, is there something stopping you from reading the data directly?
profile picture

abegy

#13
The QuoteCenter package provides interface for a DDE connector. This feature allows anyone to download all Real time data, all historical data and all fundamentals data availables with your subscription.

It's that why I will interest to develop interface with Wealth-Lab to implement strategy with a mix of analysis (Fundamentals and Technical).

As Wealth-Lab Fundamentals API is not available for anybody, can I request you to create a quotecenter extension by given you technical specification ?
profile picture

Eugene

#14
OK but again, DDE too has absolutely no connection to the Database provider.

As our support policy does not cover single-user solutions, there are two options:

1) Customize the streaming Excel/OOo DDE provider (open source, available HERE) to fit your data provider API specification (or hire a developer). The static part which will be the essential component of getting Fundamental and price data is just a dummy, so it's going to be a pretty sizable effort.
2) Wait for the upcoming Wealth-Lab MarketPlace and place your request there for a developer willing to take the job to pick it up.

P.S. Feel free to drop the specs in a ticket, don't want to give any false promises as my schedule is nearly full for 2011 yet it never hurts to take a look and make some adjustments (if/when possible).
profile picture

Eugene

#15
QUOTE:
As Wealth-Lab Fundamentals API is not available for anybody,

Now we have an open source fundamental provider:

Analyst Ratings provider
Dowlnoad Visual Studio solution (verified customers only)

Included in: MS123 Extra Fundamental Providers