Corrupted Security History
Most of my securities - about 30 - are Australian shares (ASX), with daily price updates obtained through Yahoo Finance USA. Some others - e.g. for managed funds - are updated manually.
On my last use of Moneydance I noticed some very odd looking security history graphs, and on following up I found that those updated through Yahoo Finance were affected, but not the manually updated ones.
- New prices with trading volume “0” entered for Saturdays and Sundays of most months in the existing price histories, but only up to the 12. day of each month (never for any day past the 12.) (Does this point to a wrong date format in the price update? American - month first vs. European - day first?)
- New prices with trading volume “0” entered for some public holidays (eg. Easter Monday, New Year's Day)
- Prices with “0” trading volume entered for days well into the past (e.g. the PTM the stock documented in my example started trading on 23/05/2007) and into the future – up to calendar year 2014
- Prices have also changed for some of the other days when trading did occur – e.g. the price shown in my example for 05/11/2008 - $8.80 – is outside the high / low for the day and is equal to the all-time high for this stock
This list of problems may not be complete. I did not analyze every last number in my security histories!
I noticed that the file size of the MD database increased significantly a few days ago, even though no particularly unusual user activity took place. What happened?
I opened MD backup files going back a couple of weeks. The change apparently happened on 26/07/2012, but I cannot recall any unusual user activity on that day. What could have caused the corruption of the security history database?
How do I recover from this to a usable price history. (I guess I know the answer to this question – use the uncorrupted backup file and re-enter the new data accumulated since then.)
How can I avoid having this happening again?
I have attached a file showing the effects of the problem.
Comments are currently closed for this discussion. You can start a new one.
Keyboard shortcuts
Generic
| ? | Show this help |
|---|---|
| ESC | Blurs the current field |
Comment Form
| r | Focus the comment reply box |
|---|---|
| ^ + ↩ | Submit the comment |
You can use Command ⌘ instead of Control ^ on Mac
1 Posted by -Kevin N. on 05 Aug, 2012 11:43 AM
Hi Klaus,
Something that you could try is to import your price history from your existing txns.
First, create a copy of your existing data file in case the results are less than desirable.
Open Moneydance to the 'Securities Detail' view of your Investment Account.
Click on the 'History' button.
In the resulting form, click the 'Clear' button.
On the same form, click the 'Import' button.
In the resulting form, select the 'Import from Transactions' radio button.
Click the 'import' button.
I can not speak as to what caused this issue.
HTH -Kevin N.
2 Posted by Jon Hoover on 05 Aug, 2012 01:34 PM
Hi Klaus,
Sorry to hear about the problem you are having. I am not yet sure if your diagnostic of file corruption as the cause of your problem.
Unfortunately, Yahoo does sometimes send out bad data and your errors sound like something that would be caused by them. File corruption would be more random, it wouldn't corrupt just weekends, holidays, and only affect investments.
Could try and see if this pattern is repeated next weekend for instance? You can manually delete incorrect data by going to 'Securities Detail' in your investment account, select your security in the upper right window and then press the 'History' button.
From this window you can delete the incorrect data.
Its hard to say why your data file got bigger all of a sudden. Did you start tracking foreign exchange rates or perhaps imported a bunch of old data?
If you open your MD files from before and after in a text editing program you might be able to make a guess at what the difference could be. The raw data is a little confusing but its also semi-readable to you and I.
Please let me know how you would like to proceed with this issue.
Sincerely,
Jon Hoover, Moneydance Support
3 Posted by kjahn1 on 06 Aug, 2012 06:49 AM
Hi Jon & Kevin,
Thanks for your responses. I didn't really want to delete all history data and only rely on the transaction based prices. Manually editing the incorrect data would amount to thousands of entries probably taking me weeks, especially since not only "weekend trades" have been added, but also valid closing prices changed. What a mess!
Therefore I decided to go back a couple of weeks to my backup file and manually re-enter the transactions which have happened since then.
I lean to the theory that it was a bad data download from Yahoo. The question is what can be done to improve the process. Perhaps a way to make existing history data "read-only" and only allow new data for the period between the last update and the current time (unless this protection is manually "switched off"). Alternatively a simple process to export all history data into a separate file and isolate this from the upload process, so that history data can be wiped and reloaded if necessary. One of these features - or something better - would be nice to have, but it's probably asking too much. Well, I hope it doesn't happen too often, otherwise I'll have to look elsewhere ...
Thanks for your assistance again
Klaus
4 Posted by Jon Hoover on 09 Aug, 2012 01:43 PM
Hi Klaus,
Those are some good work arounds you propose. Fortunately, bad data from yahoo is fairly rare, unfortunately it has happened more then once, so I can't promise it won't be an issue again.
Please let us know if you have any more questions.
Sincerely,
Jon Hoover, Moneydance Support
System closed this discussion on 31 Mar, 2015 03:46 PM.