Basic workflow: How do you combine historical data with intraday data?
Author: Olekso-Monsterlo
Creation Date: 3/3/2017 12:19 PM
profile picture

Olekso-Monsterlo

#1
Hi All,

I'm totally newbie in a trading world and I have a question about workflow that you use to combine historical data with a new coming intraday data.

How I see this process ?
I use historical data for tehnical analysis, I play with indicators and staretagies and at some point I need to switch to intraday
data (current day or a few last days) to take a real decision on entry/exit point. After playing a few days with WealthLab I see two use cases:

1. We use some data provider that give historical data in a runtime, like Yahoo! Finance.
This looks like the best option because you no need to update historical data each day.
But Yahoo! Finance has some problems with data (For example, OMXS30 index has missed components https://finance.yahoo.com/quote/%5EOMX/components?p=%5EOMX)
Also it doesn't contain intraday data. That means I need to switch to program that I use for trading and continue work there.

2. We bought zipped historical data in a folder and import it to WealthLab.(Like here http://eoddata.com)
This option looks really strange as for me ... We bought historical data and each day we need collect a new data and add it to existing data set.
That doesn't look handy.

Guys, describe please your Best Practices!
How do you use WealthLab to perform technical analysis if you do real trading in other programs ?

Thanks in advance,
Oleksii
profile picture

Cone

#2
Consider Yahoo! a "quick and dirty" free end-of-day provider. See Extensions for additional integrated providers (Category > Providers). Note that all Extensions are are available free to Wealth-Lab licensed customers.

As long as your markets are covered, I recommend subscribing to IQFeed for your EOD, intraday, and streaming data and real-time trading. There's a free 1 month trial. Once you have purchased a Wealth-Lab license, Install the IQFeed Provider Extension, and follow the instructions.
profile picture

Olekso-Monsterlo

#3
Thanks Cone for your advises!

I do not have plans to be professional or daily trader.
I'm planning to use https://avanza.se for trading because it's registered in Sweden, it has bank license and thanks to this integrated in tax paying process. Also they have package with a zero fees for newbie trader at Nasdaq OMX market.
Due to this I'm planning to use Wealth Lab for technical analysis and perform trading in other application. It looks like IQFeed doesn't have Nasdaq OMX data.
profile picture

Eugene

#4
Robert, IQFeed as well as most other providers do not seem to have Nordic stock data - even Quandl. Some symbols can be downloaded through Morningstar or Yahoo providers but they lack Stockholm exchange.

Oleksii, just yesterday you were not interested in intraday or real-time data completely. What made you change objective? Both your approaches do not look too robust to me. However, I've got two ideas:

1. Our Database provider could be nicely used to merge different sources together: a historical data source for backfill and a different source of real-time for intraday/streaming.
2. A poor man's alternative is the built-in ASCII provider.

Of course both solutions are not turnkey and as such would require some programming from you. Both the ASCII data or the database(s) would have to be updated intraday by a script/app that pulls data from some source.
profile picture

Olekso-Monsterlo

#5
QUOTE:
Oleksii, just yesterday you were not interested in intraday or real-time data completely. What made you change objective? Both your approaches do not look too robust to me.


The problem is that I don't have a clear vision of what I need :)
I read a book and try do the same in Wealth Lab. At this point I understand that I need to have intraday data to make decision on entry point.
My apologies for confusion but you answer clarified a lot. If I'm not US trader and play at not popular markets then I need to build proper eco-system on my own. It's Ok for me because I have programming experience and that's become a question of time.

QUOTE:
2. A poor man's alternative is the built-in ASCII provider.

That could be my case.. :D

QUOTE:
Of course both solutions are not turnkey and as such would require some programming from you. Both the ASCII data or the database(s) would have to be updated intraday by a script/app that pulls data from some source.


Just to clarify if I right understand the workflow:
1. I get archive with historical ASCII data
2. Then I write script that download intraday data on daily basis. It could use different sources for different symbols.
3. Then I write script that mix this data with archived (from point 1)
4. Then I reload it to Wealth Lab

In a result I will get all historical data (till "yesterday")
Is it correct ?
profile picture

Eugene

#6
Yes, that's right. Unlike the Database provider which is a full-fledged solution, it will neither function as desired in Streaming mode nor in the Strategy Monitor. Additionally, there may be concurrent file access issues (see here for an example of an issue with ASCII data updated externally). It's inferior to having a single data provider but for backtesting should suffice.
profile picture

Olekso-Monsterlo

#7
QUOTE:
Yes, that's right. Unlike the Database provider which is a full-fledged solution, it will neither function as desired in Streaming mode nor in the Strategy Monitor. It's inferior to having a single data provider but for backtesting should suffice.

Ok, thanks. I have better understanding of data providers now.