News provider
Author: abegy
Creation Date: 9/14/2011 3:33 AM
profile picture

abegy

#1
WLD is a very nice trading tool for backtesting strategy but there are some missing features in my point of view.

The first one that it is very important is the possibility to backtest strategy with a Walk forward optimizer process. I know that I’m not the only one which requests this feature and hope that you will be able to integrate it in the next release. Neuroshell tool has a very nice tool for that. I hope that you will integrate a better one !

The second one is the Multi-currency capabilities. This is a nightmare to manage WLD without. This feature is important for trader like me from Europe because exchange markets have most of the time stocks quotation with different currencies (like Swiss exchange or London exchange). Please reconsider this need in your top priority. WLD is only for international trader so thinks international !

Another interesting feature could be an improvement of Neuro-Lab to be able to have additional neural network module. I had a discussion on that in a previous topic on the forum.

Just for memory, I will write you again that I would be great if you can offer broker specification or addin extension only for international customer with the appropriate security to be conforming to Fidelity requirements.

And my last wish is the possibility to have multi Provider source like I have described by using translating table file to be able to manage symbol code mutation.

If you can think also to add a provider extension to manage news, I would be great !
In any case, thank you for your tools and your great support !
profile picture

Eugene

#2
Thank you for your suggestions.

1. The Optimizer API has been open sice version 5.4. Motivated developers are free to develop an extension on their own and ask for help with the API.

Currently there's no plan to include WFO in Wealth-Lab, although I might be able to look at it late this year or early next year, after finishing the IB static/streaming provider. (Anyway, this has already been voiced on the forums.)

3. Have you already created a support ticket for your NL idea?

4. MS123 is legally bound with a non-competition agreement with Fidelity, so a broker provider (even as a code sample) is not planned and therefore will not happen.

Edit

5. We're not considering a provider to manage news at this time. A recently released fundamental provider that went open source by your request fits the bill of an example necessary to understand the principles of building it.
profile picture

Eugene

#3
QUOTE:
If you can think also to add a provider extension to manage news, I would be great !

What is your specific suggestion for a stock news feed that...

1) is accessible by XML, RSS or as a single HTML page with all available news for a given ticker
2) has a sufficient news history
3) is subscription free
4) delivers the content with high "signal-to-noise ratio" (i.e. a news item for AAPL should really be a news for AAPL and not some news for an unrelated ticker like GOOG where AAPL is barely mentioned somewhere in the text)?
profile picture

abegy

#4
I was thinking of news accessible through free RSS (like Yahoo, Google or others) to be able to do something like that : buy_flag = ContainsWords('google_news', 'title', 'Buy,Increase,Up').

It would be interesting to be able to specify :
- RSS source
- Writter
- ticker news
- ticker inside news
- Where to look (title or inside the news)
- Which words to find
- date of publication


profile picture

Eugene

#5
Sorry if I wasn't enough clear but what I was actually looking for is a pointer to some particular web pages or feeds (URLs) which fully comply with the 4 bullet points above.

Given some spare time, then I might look at creating a new fundamental data provider (on a low priority basis) delivering news headers as fundamental items, if it's feasible/practical.

If you like to be able to specify it to that extent, please consider hiring a developer for that custom solution (not covered by our support policy).
profile picture

abegy

#6
I see that [LINK]http://www.marketwatch.com/investing/stock/ALU/news?CountryCode=FR[LINK] has more than 2 years online news for free and with international databases.
profile picture

Eugene

#7
Oops, fails on 1 (dynamically loaded content using Javascript) and 4 (too noisy e.g. see AAPL).
profile picture

abegy

#8
profile picture

Eugene

#9
Not sure if you noticed but we already have developed these two solutions:

Earnings Date Helper
Get symbol data from Finviz.com (company name, sector, industry, next earnings date)
profile picture

abegy

#10
Ok but Finviz is not an international solution. I don't find any symbol from Euronext France for example.
If you update your components, please do not forget to give the possibility to manage the symbol code (ALU for Euronext France is FR:ALU with MSN and ALU.PA with Yahoo).

You have the following website which are better than previous : http://www.fulldisclosure.com/dividend.asp?client=cb
profile picture

Eugene

#11
The original request was managing news and now you change the topic to earnings calendar and then again to dividends (btw available at Yahoo). Since every other post of yours is a feature request, let's concentrate on managing news first if you still need it? :)
profile picture

abegy

#12
Yes, you are right ! I prefer to continue in private for the news provider.
But, if it's possible, I'm very interesting if you can switch your current Finziv add-on to the earnings.com website
profile picture

Eugene

#13
CODE:
Please log in to see this code.

Well, it's not really a private discussion, not a custom solution, so let's please keep where it started -- here on the forum.
QUOTE:
I'm very interesting if you can switch your current Finziv add-on to the earnings.com website

The code is open so please feel free to modify it. If you have questions with that, fire away (in a new thread).
profile picture

abegy

#14
Source of news can be Reuters website which provides News (see http://www.reuters.com/finance/stocks/IBM.N/key-developments).

Except that, you have also directly exchange like NYSE which provides history database of SEC Filings (see http://secfilings.nyse.com/files.php?symbol=IBM) but it's limited to NYSE Stocks.

Sorry but I don't see any others source of news.
profile picture

Eugene

#15
#1 That Reuters source seems to be a very "low noise" feed, thanks for the key developments link. Not sure how deep would I like to go with regard to those "Next page" links though. This so far is number 1.

#2 Anyway, together with your Google RSS feed where the history depth is customizable by now we have two candidates:

http://www.google.com/finance/company_news?q=NYSE:ZMH&output=rss&num=100 (although its "noise" is higher than Reuters)
profile picture

abegy

#16
Direclty on Reuters website, you have history back to 2009 (see last page number 38 for IBM)
profile picture

Eugene

#17
My concern about going through the pages is slowing the download time down and not knowing where to stop (and no way I'd be adding a GUI control just for that single option - that would conflict with my design principles).

A downside of Reuters' news is that their key developments are in fact neutral. Not a big deal of course, just it won't be possible to codify a news as positive/negative or bearish/bullish. There's no "sign" in the newsfeed.
profile picture

abegy

#18
Agree with you. But I haven't found a free database source of news from another international providers.

What I have seen also is that most of others trading tools use only Real time news from RSS feed without database history. It means that you can trade live news but without the possibility to backtest the strategy.
profile picture

Eugene

#19
QUOTE:
Direclty on Reuters website, you have history back to 2009 (see last page number 38 for IBM)

Hey Alexandre,

How did you manage to find out the last page number for IBM, #38? Couldn't see that on their website unless I manually typed the number in the browser address bar? (BTW now the last page has become 37). Otherwise we're seriously limited to latest dozen news items with Reuters.

p.s. Have you already tried out the Classification groups feature you requested?
profile picture

Eugene

#20
Meet our first news provider for Wealth-Lab 6:

Google Finance RSS feed in

MS123 Extra Fundamental/News Providers
profile picture

Cone

#21
Awesome job here Eugene. This is really a milestone for Wealth-Lab!
profile picture

Eugene

#22
Thanks. We've just started with news and sentiment. More to come (and hopefully a TSL someday).

Just for record, here are some sources that didn't make it (failed on either one of my criteria set from 9/29/2011 4:39 AM):

http://feeds.finance.yahoo.com/rss/2.0/headline?s=symbol&region=US&lang=en-US
http://www.smartmoney.com/rss/tickernews.cfm?ticker=symbol
http://www.streetinsider.com/freefeed.php?ticker=symbol
profile picture

Eugene

#23
Customer asked and Wealth-Lab listened: the Reuters news feed is now available as well.

Install the MS123 Extra Fundamental/News Providers extension or update it to v2012.01 to power up your analysis with this high "signal-to-noise" historical news source!
profile picture

Eugene

#24
A couple of tech notes on the new Reuters historical newsfeed. Its speed is not as fast as Google's (that just has to read one RSS feed which is equivalent to an XML/HTML page). The provider has to traverse through multiple HTML pages. The number of downloadable pages is capped at 40 (AFAIR) but downloading stops when the provider detects there's no more HTML pages with news returned by the feed. Downloads are processed in a multi-threaded, asychronous manner to speed things up.