There were quite a few duplicate, even triplicate, Morningstar fundamental items as seen on charts so I decided to re-download the data, as follows:
- Closed WL7
- Deleted FundamentalData subfolder for MorningStar
- Opened WL7
- Tried to download event data for Morningstar
- Within a second the Update Log shows Event Provider Update Completed but no data was actually downloaded
Clearly there is a setting somewhere that I need to modify/delete so that WL7 can re-download the data. Please advise how to fix.
- Closed WL7
- Deleted FundamentalData subfolder for MorningStar
- Opened WL7
- Tried to download event data for Morningstar
- Within a second the Update Log shows Event Provider Update Completed but no data was actually downloaded
Clearly there is a setting somewhere that I need to modify/delete so that WL7 can re-download the data. Please advise how to fix.
Rename
Please delete the EventsRequestRegistry.txt file in the Morningstar subfolder and restart WL7 to complete the reset of accumulated fundamental data.
Did that, it still doesn't download.
Try to chart the symbol or update the DataSet first.
I need to d/l event data for *all* symbols afresh. Will test tomorrow after I d/l price data on all.
On the same topic...
It would be awfully nice if one had the option to Delete Local Files/Clear Internal Request Tracking Info for Event Providers just as it exists for Historical Data Providers.
On the same topic...
It would be awfully nice if one had the option to Delete Local Files/Clear Internal Request Tracking Info for Event Providers just as it exists for Historical Data Providers.
QUOTE:Let's decide when it becomes really necessary. Haven't heard about triplicate Morningstar event times before. Could be a one-time something.
It would be awfully nice if one had the option to Delete Local Files/Clear Internal Request Tracking Info for Event Providers just as it exists for Historical Data Providers.
QUOTE:
Haven't heard about triplicate Morningstar event times before.
((I restored the Fundamental subfolder just so I could plot the fundamentals and show duplicates/triplicates are not uncommon))
Shares out; notice the triplicate value (for a future date, no less):
Dividends; I picked just a few to show, you can see more in the image:
((Deleted the Fundamental subfolder after this exercise, will re-download afresh))
In the case of "Shares", obviously it's changed by Morningstar periodically. The provider adds those values to the BarHistory.EventDataPoints. Despite Wealth-Lab 7 deduplicates events (fundamental items) automatically, here their values are different so they appear as "new" items and get added again. That's expected.
Question is, why some dividend values got duplicated while having identical date/value pairs whereas other were not. Let's keep an eye on this to better understand the root of the issue and potential solutions.
Question is, why some dividend values got duplicated while having identical date/value pairs whereas other were not. Let's keep an eye on this to better understand the root of the issue and potential solutions.
With the Fundamental data folder & EventsRequestRegistry file deleted I started WL7 and updated Historical Data just now. Then tried to update Event Data but nothing was downloaded and the log showed Update completed:
What to do now? I need the event data, please advise!
What to do now? I need the event data, please advise!
There's nothing to update. The events data has already been collected behind the scenes when you updated the historical data.
Not true; while the FundamentalData folder got created it was empty.
However, I solved it... see next post.
However, I solved it... see next post.
Hey Sammy, when you clear out all of the Events data, the next time you try to update them from the Data Manager, nothing will happen. That's because there are no existing event data to update. The Data Manager only updates data for symbols that you've already downloaded at least once on demand.
WL7 updates all selected providers on demand when you perform a backtest. The idea is you needn't worry about manually updating data. After you run a backtest, or even just open a chart, with an Event Provider enabled you should see the Events. And the next time you update the Events from the Data Manager, those symbols should now update if there is new data.
The reason it works this way is because you might have historical data for thousands of symbols stored locally, but only use Event data for the Dow 30, for example. It doesn't make sense to bog down the system downloading thousands of Event data that you'll never use.
The original idea from Quantacula Studio is to let the data updates happen in the background as needed. In fact, there was no Data Manager at all. But due to the expectations of the WL user base having a Data Manager we went ahead and added it to WL7, but it's optional and in fact I rarely use it.
WL7 updates all selected providers on demand when you perform a backtest. The idea is you needn't worry about manually updating data. After you run a backtest, or even just open a chart, with an Event Provider enabled you should see the Events. And the next time you update the Events from the Data Manager, those symbols should now update if there is new data.
The reason it works this way is because you might have historical data for thousands of symbols stored locally, but only use Event data for the Dow 30, for example. It doesn't make sense to bog down the system downloading thousands of Event data that you'll never use.
The original idea from Quantacula Studio is to let the data updates happen in the background as needed. In fact, there was no Data Manager at all. But due to the expectations of the WL user base having a Data Manager we went ahead and added it to WL7, but it's optional and in fact I rarely use it.
Solved!
The way to force Event Data updates is *not* via the Event Providers tab but via the DataSets tab... click on a dataset then click on Update DataSet button... this forces *both* a Historical as well as Event data update.
Unless one has a consolidated dataset with *all* the symbols one will have to update each dataset individually to force this process. :(
Here is my updated dividend history for AAPL:
(Compare with previously posted image, above)
No more duplicates or triplicates (yay!) - for now anyway - but there's a 17-year gap (??) I've highlighted, the developers may wish to look into it (perhaps block or delete any data over 10 years old... most free providers won't offer data that far back anyway, I know that Fidelity limited its fundamental data to about 5 years or so).
My final thoughts follow...
The way to force Event Data updates is *not* via the Event Providers tab but via the DataSets tab... click on a dataset then click on Update DataSet button... this forces *both* a Historical as well as Event data update.
Unless one has a consolidated dataset with *all* the symbols one will have to update each dataset individually to force this process. :(
Here is my updated dividend history for AAPL:
(Compare with previously posted image, above)
No more duplicates or triplicates (yay!) - for now anyway - but there's a 17-year gap (??) I've highlighted, the developers may wish to look into it (perhaps block or delete any data over 10 years old... most free providers won't offer data that far back anyway, I know that Fidelity limited its fundamental data to about 5 years or so).
My final thoughts follow...
My final thoughts:
- Event Data errors happen, and accumulate over time
- therefore, it's good practice to periodically flush the data and download it afresh to minimize them
- the developers can help in this process - and make it as intuitive, easy and error-free as possible - by having the option to Delete Local Files/Clear Internal Request Tracking Info for Event Providers just as it exists for Historical Data Providers
Just my 2 cents.
- Event Data errors happen, and accumulate over time
- therefore, it's good practice to periodically flush the data and download it afresh to minimize them
- the developers can help in this process - and make it as intuitive, easy and error-free as possible - by having the option to Delete Local Files/Clear Internal Request Tracking Info for Event Providers just as it exists for Historical Data Providers
Just my 2 cents.
Oops, Glitch... I was creating my posts while you were writing yours lol
QUOTE:
The way to force Event Data updates is *not* via the Event Providers tab but via the DataSets tab... click on a dataset then click on Update DataSet button... this forces *both* a Historical as well as Event data update.
This is exactly what I implied when saying this...
<<There's nothing to update. The events data has already been collected behind the scenes when you updated the historical data. >>
And here's what you had said: <<I started WL7 and updated Historical Data just now.>>
QUOTE:
17-year gap (??)
Well, this is an unofficial connector to the free data feed. 🤷♂️
No worries! And yes, the DataSet update does the same thing as the on-demand update from a backtest or chart request, so it will update Event data too for the selected Providers.
@Glitch:
Some of my strategies run on large datasets (1000's of symbols) so I like to have the data already downloaded rather than do it on the fly, much quicker this way I believe.
QUOTE:
WL7 updates all selected providers on demand when you perform a backtest
Some of my strategies run on large datasets (1000's of symbols) so I like to have the data already downloaded rather than do it on the fly, much quicker this way I believe.
One last point:
Actually, even when you have previously downloaded data some symbols in a dataset just don't update while others do, as shown here:
The only workaround - which works sometimes, not always - is to update each dataset individually to force these stragglers to update. The image above shows 2 symbols still not updated after a Historical + Dataset update. Not sure why this happens.
[Thought I'd mention it here even though this thread is about Event Data. Apologies if I did wrong.]
QUOTE:
The Data Manager only updates data for symbols that you've already downloaded at least once on demand.
Actually, even when you have previously downloaded data some symbols in a dataset just don't update while others do, as shown here:
The only workaround - which works sometimes, not always - is to update each dataset individually to force these stragglers to update. The image above shows 2 symbols still not updated after a Historical + Dataset update. Not sure why this happens.
[Thought I'd mention it here even though this thread is about Event Data. Apologies if I did wrong.]
If a symbol update fails for some reason, WL7 won't attempt another update until the following day. This is to avoid continued and persistent lags that occur when WL7 keeps trying to update "problem" symbols. Updating the DataSet like you're doing clears the blacklist data. You can achieve that same thing by right clicking the provider and selecting "Clear Internal Request Tracking Info".
The image that I posted is for an Unlinked dataset so there's specific provider whose tracking info can be reset. As you said, one has to wait until the next day.
A trick that seems to work is to open a chart of the symbol with the Offline Mode unchecked - but that's too cumbersome when there's a bunch of affected symbols.
A trick that seems to work is to open a chart of the symbol with the Offline Mode unchecked - but that's too cumbersome when there's a bunch of affected symbols.
Your Response
Post
Edit Post
Login is required