GICS in WLD? Fundamental data for Chinese stocks?
Author: tradercn
Creation Date: 11/4/2013 10:57 AM
profile picture

tradercn

#1
I did not see GICS.xml under data directory, is there anything wrong? Or, I just guess, GICS might not be supported by WLD.
If, just in case, GICS is not supported by WLD, do you have any idea to implement it?
It sounds fundamental analysis for Chinese Securities is not integrated in WLD as well, so I have to make my own database for that? Is it?
Thanks.
profile picture

Eugene

#2
1 - GICS has never been supported in WLD: GICS Classification Groups is a WLP-only feature.

2 - There are no data providers offering fundamental data for Chinese securities.
profile picture

tradercn

#3
Thanks for your quick answer.
I wish WLD provides GICS and fundamental analysis API later. :)
profile picture

Eugene

#4
Sorry, there has never been no plans to support GICS in WLD.

As for fundamental analysis, if you have a data source that fits, have you considered building a fundamental data provider around its API?
profile picture

tradercn

#5
That is because I start to think to mix fundamental and technical analysis. For example, during sector rotation, I wish to pick stocks which average PE<20 in different sectors, by weakest RSI.
profile picture

Eugene

#6
The question is, do you have a fundamental data feed with the API for Chinese stocks?
profile picture

abegy

#7
Eugene,

It could be interesting to create the possibility to request for a stock the "Sector, Industry and Sub-industry" based on the Bloomberg website (http://www.bloomberg.com/quote/BNP:FP).

Or, maybe, give us the possibility to create the "GICS files" for WLD customer based on what you have developped for the BBFree extension (which offer to select stocks by these criteria).

As you have understood, I'm interesting too ;-)
profile picture

tradercn

#8
Hi Eugene,

I have a data feed which updates price and major financial data except sector information which could be created manually.

As far as I know, all Chinese stocks are categorized by GICS, Provinces, and Concepts. Take Concepts as an example. Stocks are categorized by different concepts, such as Shanghai Free Trade Zone, High Speed Railway, Gene Chips, Urbanization and etc. These concepts are updated frequently and manually. Provinces and GICS are updated very rare. I try to make a database for the above.

Chinese data feeds follow Tongshi API standard. What I am using currently is WangJiFeng EOD. Its web address is www.51wjf.com. FYI.
profile picture

Eugene

#9
Hi Fan Zhang,

Wish I could help but I'm not familiar with Chinese GICS and data feeds and don't know a thing about Tongshi API. Add language barrier here. Sorry.

Hi Alexandre,

Sounds like a good idea for a D.I.Y. project that I leave up to you. Feel free to build your GICS classification using the files already downloaded on your PC by the provider. They can be found in the B....gProvider subfolder under \AppData\Roaming\Fidelity Investments\WealthLabDev\1.0.0.0\Data\

Stocks-SectorCodes.txt
Stocks-Symbols.txt
Stocks-CountryCodes.txt

Each sector from Stocks-SectorCodes.txt has a unique code assigned that can (or can not) be found in Stocks-Symbols.txt. The first column is symbol name, the 2nd column is country code (mapped to Stocks-CountryCodes.txt) and the 3rd and last column contains sector code that can be directly matched to Stocks-SectorCodes.txt. Sounds really easy, so good luck.
profile picture

abegy

#10
Thank you very much Eugene !
profile picture

tradercn

#11
Eugene, please don't say sorry. I know you tried your best.
Thanks anyway.
profile picture

superticker

#12
QUOTE:
GICS has never been supported in WLD: GICS Classification Groups is a WLP-only feature.
So if WLP is being discontinued at the end of the month (June 2020), are there now plans to support GICS on WLD? And will that solution be available by the end of June 2020?
profile picture

Eugene

#13
Sorry, there has never been plans to support GICS in WLD. Feel free to correct me if I'm wrong but my understanding is that it requires substantial licensing fees.
profile picture

superticker

#14
Well, is there a data feed provider where I can extract the GICS codes for each stock to create my own GICS lookup dictionary?

I'm only asking because if US users have to move from WLP to WLD, I want to know how to do GICS lookups on the WLD platform.
profile picture

Eugene

#15
Yes, there is MorningstarSymbolDataExtractor class in Community Components which can be used to look up an instrument's sector, industry and NAICS / SIC / ISIC codes:

Get symbol data from Morningstar.com (sector, industry, code etc.)

Supported are both U.S. and international stock symbols.
profile picture

superticker

#16
Looks interesting; thanks a bunch. I wonder if this class does data caching or not? Hopefully it does.
profile picture

Eugene

#17
You're welcome. Data caching in this class has never been on the table. Nothing stops you from implementing it on your own - the solution is open source ;)
profile picture

superticker

#18
QUOTE:
Data caching in this class has never been on the table. Nothing stops you from implementing it on your own
I might implement something eventually. I'm thinking of date stamping each entry and when it's two years old refresh it (i.e. remove the instrument symbol if it's delisted.).

What would be nice would be to use data from the Fidelity side somehow, but that's probably password protected. I wonder how difficult it is to do authentication with .NET?
profile picture

Eugene

#19
Makes sense. Follow examples in EarningsDate.cs, UpcomingSpinoffs.cs or UpcomingSplits.cs in the open source code of Community Components. This should make it easy to implement it.

With regard to GICS, what data from the Fidelity site do you mean specifically?
profile picture

superticker

#20
On the GICS codes caching issue, since there are multiple Chart windows and instances of the Morningstar class, the cache would need to be write-through to SetGlobal so it stays clean. I can't afford dirty pages in the cache with multiple instances about; that's too complicated to manage. For the first instance, the constructor would load the cache from disk. On the closing of the last instance (so we need an instanceCounter), the destructor would copy the cache global to disk when the last Chart window closes.

Of course, if WL ABENDs (e.g. power failure), the destructor is never called, so this isn't a perfect solution. It won't take much coding (the WL and .NET frameworks do all the work); use Dictionary<symbolKey,structRecord> to store everything by value in SetGolbal. But it's still a complicated implementation that can't be done in a weekend.

Oh there's a problem. When the Dictionary collection grows, there's going to be garbage collection problem for SetGlobal and the write-through cache. How do a work around that without doing extents (or journaling)? I could allocate extra slots when declaring the collection. How many extra slots should a collection declare?

All I want from Fidelity is the GICS codes for individual stocks. I don't know if there's a server (and RESTful API) that supplies that, but if there is I would be interested. There would still be an issue with the authentication for service access.
profile picture

Eugene

#21
If I have some free time I might help with scraping a page with GICS codes for a stock or a group of stocks. This page, if exists, must not require authentication.
profile picture

superticker

#22
QUOTE:
scraping a page with GICS codes for a stock or a group of stocks. This page ... must not require authentication.
This is the only Fidelity page I know about that returns Sector and Industry information on a stock. See Company Profile at https://eresearch.fidelity.com/eresearch/goto/evaluate/snapshot.jhtml?symbols=aapl

Maybe we are moving too fast with this. Shouldn't someone ask Fidelity if there's a "formal" API (AJAX call?) for doing this directly before contemplating scraping? Gee, this is for Fidelity customers, so Fidelity should provide an API, which would be more efficient than scraping.

How does the WLPro provider do it? BTW, that approach broken down in 2017 when they switched to secure connections. A fix was suppose to come out for WLP 6.9.23.
profile picture

Eugene

#23
Anything but this simple scraper would go beyond the quick help I intended to provide here. To make this work, check System.Xml in Wealth-Lab's References dialog (.NET Framework tab). Good luck.

CODE:
Please log in to see this code.
profile picture

superticker

#24
Thanks for all your help. I still think Fidelity should provide an API to this data directly (without scraping). I've done websites and used the Firefox debugger over the DOM, but I've never scraped before.
profile picture

Eugene

#25
In Community Components v2020.07, made some cosmetic fixes to the MorningstarSymbolDataExtractor class.
This website uses cookies to improve your experience. We'll assume you're ok with that, but you can opt-out if you wish (Read more).