Extracting Shares Related Info for Proshare ETFs
Author: Harapa
Creation Date: 8/30/2015 8:25 AM
profile picture

Harapa

#1
Having seen the code for downloading Symbols http://www.wealth-lab.com/Forum/Posts/Stocks-in-Russell-1000-and-Russell-3000-35500, I wonder if one can write similar code to extract shares outstanding & AUM from this link and best if such a code can use symbols from a data set, and save the data as .WL data file.

https://accounts.profunds.com/etfdata/ByFund/SSO-historical_nav.csv

In my practice, I append AUM as OHLC and shrs as volume in one file.

I am sure many others will be interested in it.

TIA
profile picture

Cone

#2
Anything is possible, but it's unnecessary work for zero gain.

The file is perfect as csv for the ASCII provider, where you can specify the AUM field as Volume. In the Data Manager > ASCII Data tab, turn on "Cache ASCII Data" and the file will automatically be converted to WL format in cache whenever the csv file is updated.
profile picture

Eugene

#3
Here you go. This code downloads the CSV and parses it on-the-fly. I leave saving to .WL file to you as coding this is explained in the QuickRef and creating a DataSet in the Wiki FAQ > Data, respectively. Or as Cone says, it's pretty pointless. You might also want to consider this technique: Access Data from the Internet on-the-fly.

Click "References..." in the Strategy Editor, then select "System.Core".

CODE:
Please log in to see this code.
profile picture

Harapa

#4
Thank you Eugene.
I had looked at the link but was not able to see where I specify which header to import. Between this and WIKI example I will learn that part.

Cone, I will work on your suggestion, as I can download/save many data files in a zippo using a macro in Excel.
profile picture

Harapa

#5
QUOTE:
The file is perfect as csv for the ASCII provider,

So I downloaded data for over hundred symbols (using my Excel Macro) and saved these as .csv When I import these via ASCII provider all data files are truncated after 1/28/2015 (this truncation is visible in the preview window of ASCI Data provider as well as on a Chart in WL). To share with you, I created one of the files as .txt which when imported also truncates after 1/28/2015. Looking at many files in note pad (where format errors are easier to spot) I couldn't see anything obvious.
Please try the enclosed file to replicate the issue at your end and tell me if you see the same and what can be causing it.
Thanks for you help.
profile picture

Eugene

#6
Obviously your settings are wrong i.e. you configured the ASCII provider incorrectly.
profile picture

Harapa

#7
QUOTE:
Obviously your settings are wrong

Clarify, can you import data till 8/28/2015 using the above file...
profile picture

Eugene

#8
For the discussion to stay productive, please provide screenshots of your ASCII DataSet configuration and Preview window. Use its "Copy format from a DataSet" feature to quickly rerpoduce the format used, or attach the DataSet's XML file.
profile picture

Harapa

#9
QUOTE:
"For the discussion to stay productive, please provide screenshots of your ASCII DataSet configuration and Preview window. "

Attached

QUOTE:
"Use its "Copy format from a DataSet" feature to quickly rerpoduce the format used,"

Can't figure out this function
or
QUOTE:
"attach the DataSet's XML file."

.pdf of file is attached as Forum doesn't give an option to attach .XML


On a side note: A data file generated using your code also produces the same error ( off course the "field order" is chosen to fit the layout of that file)
profile picture

Eugene

#10
The Date format is incorrect, "mm" stands for minutes. Should be "MM" for months.

Both that and "what you couldn't figure" are referenced in the User Guide: Data > Data Manager > Create New DataSet > Data Providers > ASCII > Format Options.

P.S. XML could be zipped to attach.
profile picture

Harapa

#11
That fixed it. It is the last thing you will think of when 90% data reaches the destination correctly. Thanks for your help.

Would you mind if I were to post VBA code and instructions on how to download data for all instruments using Excel?"
profile picture

Eugene

#12
Thanks but we're not an Excel board, and my code already does just that (you just need to add the file save capability which is trivial and exemplified: Wiki > Exporting to ASCII comma-separated files). Everybody can use their own download tool, be it WLP, wget, curl, some download manager, or even Excel.