How to create own Static and RT data providers?
Author: orfeus2006
Creation Date: 2/9/2010 3:27 PM
profile picture

orfeus2006

#1
Hi all,
As I know, there is a QuickGuide to write the subj on the WL 4.x
But I can't find any documents about WL 5.x one.
Yes, I've seen the Community.Providers project, but it's too small to understand abilities and it's realization.
Could anyone help me with source code examples of creating real working Static and RT Data Adapters, based, for example on RDBMS (MS SQL or any) and small timeframe (1 second).

Best Regards,
Alexey
profile picture

Eugene

#2
profile picture

orfeus2006

#3
Hello, Eugene
There is just one little problem - I'm not from USA and could not be the Fidelity customer.
I'm trying to understand what platform should be based my algotrading on. WLD 5.6 is good one, but without API documentation it looks like very risky.

profile picture

Eugene

#4
Hi Alexey,

I realize. Wasn't suggesting you to call Fidelity though. As for me, this question is so widely requested that I have a trail of support tickets asking for the API docs, and have put an item in the Wealth-Lab 5 Wiki FAQ.

Currently, there's nothing to add/offer above what's already said in the thread - we've been waiting for the release, and looks like we have another promise of it.
profile picture

Eugene

#5
profile picture

Eugene

#6
I'm developing a provider for SQL Server databases for Wealth-Lab 5.6 and higher. Support for any ODBC/OleDb compliant sources is considered but not yet carved in stone.

Anyone interested in using SQL data in Wealth-Lab 5 is welcome to leave their ideas and suggestions here or in a new, dedicated thread.
profile picture

DaveAronow

#7
Eugene,

Why not do a provider for ADO.net instead, then ado.net data provider could be used, not just sql server?

Dave
profile picture

Eugene

#8
Hi Dave,

ADO.NET will be used. Basically the use of SqlClient for SQL Server and OleDb for the rest seems reasonable because they are optimized for their tasks.
profile picture

DaveAronow

#9
Eugene,

As an example yes those would be helpful. However if you make it more dynamic you could have it load the provider type and provider configuration settings from a config file (similar to what you'd do with spring/spring.net) and then you can have it link to any database accessible via ado.net (for example sqlite, dotnetfirebird, oracle, mysql, etc).

If you have the adapter set to run against any ado.net implementation that implements System.Data.DbProviderFactory you could then have something that will work with all of the above mentioned implementations.

As a start though I'm sure sql server and odbc would also be very useful.

Dave
profile picture

Eugene

#10
Dave, thank you for these suggestions - there's some food for thought.
profile picture

bobydesi123

#11
Eugene,
u said "I'm developing a provider for SQL Server databases for Wealth-Lab 5.6 and higher".Will it be an open source provider? Would I be able to use LINK feature in .NET 3.5 or will it still be a .net 2.0 environment.
profile picture

Eugene

#12
Re: open source - haven't decided yet, but if there's customer interest in making the code open source, we'll consider doing so.

Re: .NET 3.5 - it will be interesting to hear why'd you need LINQ support?

The SQL/ODBC datasource provider is envisioned as a user friendly, easy to set up tool. Think of creating a DataSet in the manner of the ASCII provider's Field Order / Format Options dialog, but simpler.

The idea is to get some core functionality delivered relatively quickly, so its first release may even lack support for custom data series and streaming data (still researching), and add extras and tweaks later by request.
profile picture

jatubio

#13
Hello,

Eugene, i'm very interested in one open source ODBC datasource provider. I think if it's opensource, anybody can to help to make the best datasource provider ;)
profile picture

Eugene

#14
Hello Juan,

As said above, I'm yet undecided but to highlight, per Fidelity request and according to our own experience, the open source code of a streaming provider will be disclosed to known customers only.
profile picture

Eugene

#15
Hi Dave,
QUOTE:
If you have the adapter set to run against any ado.net implementation that implements System.Data.DbProviderFactory you could then have something that will work with all of the above mentioned implementations.

Thank you for your expertise. I decided to implement a universal static/streaming database provider exactly the way you suggested, supporting any ADO.NET compatible factory.

Furthermore, for MS SQL Server it will have special support for asynchronous operations.
profile picture

DaveAronow

#16
Eugene,

How can we get a copy when it's released -- can we open a support ticket? I'm having some nice debugging fun with my streaming adapter and I'm thinking I missed some required piece.

Dave
profile picture

Eugene

#17
You're welcome to our support portal, Dave.

This month I'm planning to let the DDE streaming provider go open source (to known customers, as per Fidelity's request.)
profile picture

Eugene

#18
Thought I'll give an update here. There's not going to be a standalone MS SQL Server provider. Instead, it will become a part of the Database provider capable of working with any ADO.NET compatible sources. The MS SQL client will still have asynchronous operation though (kind of performance optimization). We plan to release this provider (and other streaming providers the community's been waiting for) after the upcoming version of Wealth-Lab is released later this year, fixing some issues with international streaming data.
profile picture

jrm999

#19
Hi Eugene, by any chance do you have any updates on when the Database provider will be available? I suspect there are a lot of moving pieces behind the scenes before it can be released. Was just checking (I'm really looking forward to using it).

Thanks! Ray.
profile picture

Eugene

#20
In about a week. The initial release is ready, and recently I have put up sort of an online manual in the Wiki:

Database static and streaming provider
profile picture

jrm999

#21
The document is great! (Along with the companion example for bulk import of data in CSV files to SQL). I'll use these in the meantime to go ahead and get my database populated (and work on code to populate it in realtime) now.

Thanks again for your quick response!

Ray.
profile picture

Eugene

#22
Thanks for the good words.

It striked me that releasing the static provider only perhaps is the best option, considering that version 6.1 (hopefully) brings some fixes which would render the provider's "Default static data source" pane obsolete (it wasn't intentional, just a workaround for some shortcoming.) So maybe it's better wait with releasing the streaming part.
profile picture

jrm999

#23
Is there an estimated date for when 6.1 would be released? The main thing I'm looking for is the ability to load futures data as close to real-time as possible, and view the data in the charts with various indicators.

I already have a process setup to read data (Time, O, H, L,C, Volume) each second for a small list (currently 5) of contracts and was writing it to a flat-file. I was planning to modify the write logic to update a set of SQL tables (or perhaps update a single table that also has the symbol as a column, and then create a view for each contract that I would then use in Wealth-Lab). If I still did this write to SQL, and we just had the static provider, would I need to do anything in WLP to retrieve the data from the most recently updated rows? I experimented several months ago with writing my data to text files, and if I remember correctly, WLP would update the chart whenever the file was updated.

If I'm making this harder than it should be, please let me know.

Thanks again,

Ray
profile picture

Eugene

#24
Ray,

Just like tables, views will be supported by the Database provider. To retrieve the static data from a table or a view, you just load/refresh the chart in WLP (which gets updated every time the "Go" button is clicked if there was a change to the database.)

Not sure if your question was related to retrieving live (streaming) data though. It will be possible with the provider's streaming part, which most likely gets delayed until 6.1 (no ETA).
profile picture

Eugene

#25
With the recently released Database static/streaming provider, connecting to database have become effortless.
profile picture

jrm999

#26
I've started using the Database provider, and wanted to say how much I appreciate the work done to make this available. So far I've just added data for a few futures contracts, but plan on expanding to more contracts, plus a real-time feed.

Ray