Is there a way to reload chart history via an API
Author: bgood
Creation Date: 8/25/2009 10:16 PM
profile picture

bgood

#1
I have written a form of bad data checking that will scan a complete data set and generate a list of symbols with bad data. I have then generated a new dataset with just the bad symbols (> 300). Now rather than loading each symbol and RightClick->ReloadSymbolData, I would like to write a simple script to reload the data. Is it possible to do the data reload via an API?
profile picture

RA81

#2
yes. you can do it via API. official method is to use
CODE:
Please log in to see this code.


I have a different way, but it only works if your strategy is framed in a library (dll)
profile picture

bgood

#3
I was under the impression that GetExternalSymbol() only read the data that was already on my computer and that it did not force a new download of the data. What am I missing?
profile picture

RA81

#4
QUOTE:
I was under the impression that GetExternalSymbol() only read the data that was already on my computer and that it did not force a new download of the data. What am I missing?

I think, If you check "UpdateDataOnDemand" then data will be updated every time :).
profile picture

Eugene

#5
Bryce,

It's hard to see how this little code that I once wrote for QuickRef or On Demand Update could be useful for your task. I think RA81 confuses updating a DataSet with refreshing the data.

To refresh means to find the symbol(s), delete their files, download the data from scratch and save them as Bars. I probably can think of a way to make it in a strategy using the static API.
profile picture

bgood

#6
Thank you Eugene. I will wait for your update regarding this API.
profile picture

RA81

#7
QUOTE:
I think RA81 confuses updating a DataSet with refreshing the data.

ops... if bghood want to reload all data (not only update) then Eugene is right.
But (I don't check this) don't WLD code reload data from dataSource every time when data is requested?? If "cache data" is on then WLD try to load it from cache, otherwise from source. Thomething like this i think. (Metastock and ASCII adapters work so).
profile picture

Eugene

#8
I believe that Bryce is talking about Fidelity data. Neither Metastock/ASCII adapters have the possibility to Reload Symbol Data, nor it makes sense in this context.
profile picture

RA81

#9
QUOTE:
Neither Metastock/ASCII adapters have the possibility to Reload Symbol Data

If i click on the symbol in metastock dataset then all data are reloaded for this symbol, right? So if he uses not online data but something like Metastock static provider, then refresh and reload are equal, I think. or I don't understand what is it - "Reload Symbol Data"
profile picture

Eugene

#10
Refer to the Wealth-Lab User Guide, Charting > Chart Context Menu > Reload Chart History.
profile picture

bgood

#11
Hi Eugene. You are correct that I am talking about Fidelity data.
profile picture

Cone

#12
I think the question is if you can connect to the Fidelity Server API yourself. No. Even if you want to take a hack at it and succeed, you'll have to decrypt it next.
profile picture

bgood

#13
I suspect that everyone really understands what I am trying to do, but to be sure I will elaborate a little. I have written a scanner to sort through a large number of stock looking for an interesting "event". During that I discovered a large portion of the symbols stored on my system contained lots of data errors (some symbols have hundreds of errors). I used the utilities->Bad History Data Checker and soon became frustrated because of checking each symbol individually. I then modified that code to scan a DataSet to find all symbols in the DataSet that contained bad data and output the list in the debug window. I have built a new DataSet that contians a list of symbols (> 300 daily) that have bad data, and would now like to automate the "Reloading Symbol Data" to my system. Going through each symbol by hand is tedious (~75% of the 5 min data has errors in the last year). This is why we write programs to do our work:-) If there was a menu pick to refresh all symbols in the DataSet, I would be done! I assume that there could be an API that would allow me to programatically do what I can manually do (Reload Chart History for a single symbol). I would assume that all customers have large numbers of data errors and ignore them. For some of us, those errors are important to be fixed. Please let me know if more information is required.
profile picture

Cone

#14
If you've got the list of symbols, then you need only to write a little code to go through the Provider folders to kill those data files. Then, the next time you update, they'll be refreshed. Sounds like you'll be able to do that yourself, but if you need help, let us know.

Keep in mind, however, bad data may get replaced with bad data unless you tell your provider about it first.
profile picture

Eugene

#15
Going through the list of symbols in BarDataStore is what I had on my mind to try, no Fidelity API :)
profile picture

Cone

#16
Just thought of another way without using code - use Fidelity's Quarantined symbol list feature:

1. Close WLP

2. Add the symbols to kill to the ..\Data\FidelityStaticProvider\SymbolQuarantineList.txt file. Each symbol should be on a new line, and, you need to add an "=" sign to the end of each symbol. See the file for examples.

3. Start WLP, open the Data Manager > Fidelity Data tab, highlight the symbols in the Quarantine list to delete, and click the button to delete them.

Note! This actions deletes data from all time frames for each symbol, not just Daily.

----
Here's a way to programmatically add all the symbols of a DataSet to that file (just an example) -

CODE:
Please log in to see this code.
profile picture

bgood

#17
Hi Cone and Eugene,

From Cone's 6:55 post, I infer that if I delete the files in .../roaming/Fidelity Investments/WealthLabPro/1.0.0.0/Data/FidelityStaticProvider/Daily/... that have bad data, then at the next update, I will "Reload Chart History" automatically at the next update. Is my understanding correct?

From Eugines 6:58 post, I am not sure what you are thinking?

From Cone's 7:04 post, if I understand, this will cause Wealthlab to quarantine all of the symbols and delete the data. Then be removing the symbols from the quarantine list, I could update and get the data updated. Do I have it correct?

There seems to be a broader issue here that I would like to understand. How can I avoid getting bad data in the first place. I can make many assumptions regarding why bad data occurs, but I clearly lack enough details to fully understand. However, for many customers, having bad data will alter backtesting results and could cause bad decisions to be made.
profile picture

Eugene

#18
QUOTE:
...if I delete the files ... Is my understanding correct?

Yes.
QUOTE:
From Eugines 6:58 post, I am not sure what you are thinking?

I was think of a somewhat more complex scripting solution that would delete the bad symbol's files using some static provider API calls.
QUOTE:
Then be removing the symbols from the quarantine list, I could update and get the data updated.

Correct.
QUOTE:
How can I avoid getting bad data in the first place. ... However, for many customers, having bad data will alter backtesting results and could cause bad decisions to be made.

As data isn't our product, please call Fidelity when you see bad data coming in.
profile picture

Eugene

#19
Hi Bryce,

Here's another solution to wipe out your bad symbols' data. Some notes:

* Put your tickers in an array. It's ideal if you already have a line of bad symbols delimited by comma or space: then you just have to paste that line in BadSymbols.
* Designed to work with Fidelity data.
* If you're concerned about dirty daily data for a number of symbols, the code will not affect the data in other bar scales at all (e.g. 30-minute). Therefore, you need to run the code in single symbol mode on a daily chart of any Fidelity symbol. For another bar scale, paste another bunch of symbols and repeat the procedure on a chart in that time frame.

CODE:
Please log in to see this code.


Running this strategy will zero your bad symbol files. One step left is to manually update your DataSet(s) to reload data from scratch.
profile picture

Rachel123

#20
I think you should search google, You will definitely found solution for your problem.


Thanks and Regards
Rachel
profile picture

Eugene

#21
And I think that what you have done by attaching the address was a marketing trick to promote your site -- which has completely nothing to do with trading, Wealth-Lab or this question.

Please no illusions: this has never worked here and won't do.
This website uses cookies to improve your experience. We'll assume you're ok with that, but you can opt-out if you wish (Read more).