huge .moneydancesync folder (781mb)
I just cleaned up my sync situation recently (recreated sync hosts and am using Dropbox Folders). All of a sudden, one of my .moneydancesync folders is 781MB. There are a bunch of "conflicted copy" .txt files and under the v3 folder there are a handful of huge .mdtxn files, two are 771mb each.
I had been working with MD (5007) on a Mac Mini containing a sync copy of my MD data and no other copies of MD are open/running.
Should I be worried (again)?
Comments are currently closed for this discussion. You can start a new one.
Keyboard shortcuts
Generic
| ? | Show this help |
|---|---|
| ESC | Blurs the current field |
Comment Form
| r | Focus the comment reply box |
|---|---|
| ^ + ↩ | Submit the comment |
You can use Command ⌘ instead of Control ^ on Mac
1 Posted by Milo C on 06 Aug, 2023 04:15 PM
Note that this is my .sync folder that is huge. My .moneydance file is now fine, about 140mb in size.
2 Posted by Stuart Beesley ... on 06 Aug, 2023 04:28 PM
No. Don’t be worried.
Where is your dataset located?
The large mdtxn files are your whole dataset trunk files being pushed to sync.
Run toolbox extension. Update mode. Advanced. Shrink dataset. Use 1 day.
Let me know results.
After run the analyse dataset and objects feature. Upload those contents here.
3 Posted by Milo C on 06 Aug, 2023 04:59 PM
Would you mind sending me a private sharing link again, please?
4 Posted by Milo C on 06 Aug, 2023 05:26 PM
My main dataset is located in the usual Containers
location, /Users/<username>/Library/Containers/com.infinitekind.MoneydanceOSX/Data/Documents/.
I'm syncing using Dropbox Folders.
After shrinking to 1 day with Toolbox, my backup and dataset are each about
126MB.
My .moneydancesync folder is still huge. (Also, there are two folders under
there with different GUID folder names. Why are there multiple files?)
5 Posted by Milo C on 06 Aug, 2023 05:28 PM
The "conflicted copy" txt files are still in the .moneydancesync folder. They look like:
mdsyncinfo (MacBook-Pro's conflicted copy 2023-07-22 1).txt
and
mdata-v1 (MacBook-Pro's conflicted copy 2023-07-31 1)
mdata-v1 (Milo Chan's conflicted copy 2023-07-22 1)
6 Posted by Stuart Beesley ... on 06 Aug, 2023 05:41 PM
BACKUP FIRST
you can delete the conflicted txt file copies, just leave the original version.
I would turn off syncing.. then use toolbox, advanced, reset sync settings. then quit MD
Then delete all the three .moneydancesync different GUID folders. Then launch MD. Setup syncing again...This will create a new sync folder
7 Posted by Milo C on 06 Aug, 2023 05:45 PM
Question before I do this: the copy where I am experiencing this is a "secondary copy", not the master. Does this matter?
8 Posted by Stuart Beesley ... on 06 Aug, 2023 06:21 PM
What you said doesn’t make sense? You said the files in the sync folder were large? These are in the cloud and common to all datasets. Please clarify?
9 Posted by Stuart Beesley ... on 06 Aug, 2023 06:22 PM
… and if you do this then you are setting up sync from scratch. You will have to setup sync again on primary. And then on secondary use the file / new / open sync copy procedure.
10 Posted by Milo C on 06 Aug, 2023 08:02 PM
Sorry, my explanations are confusing because I don't understand how the sync really works.
Yes, the .moneydancesync folder is in Dropbox, so it is indeed a cloud folder that Dropbox is syncing between my multiple computers. One of the GUID folders inside that folder is very large, over 781mb.
So, what I'm really saying is that I don't understand why the cloud sync folder is so large. and why it continues to accumulate these "conflicted copy" versions of mdata-v1 and mdsyncinfo.txt files. Indeed, another "mdata-v1 (MacBook-Pro's conflicted copy 2023-08-06 1)" file was created less than 10 mins ago.
I have my two computers on the same desk at the moment, and I am running MD on one, closing it, letting DropBox sync, then opening MD on the other computer.
As I do this, other problems are appearing in MD. While I was using the "Secondary" copy, I created some new hierarchical bank accounts, and it looks like this confused MD a little when I opened the "Primary" copy and it synced.
An account has appeared, "Restored 20d39add-2fe6-4a65-b918-17486ff1ffc8 from txn 20230714 2023 Contribution AIP 451730", with a transaction in it that I do not recognize.
All of this is making me extremely uncomfortable. Is my MD data getting corrupted?
11 Posted by Milo C on 06 Aug, 2023 08:22 PM
Not knowing the sync architecture, I can rationalize that the sync design does not place any limitation on how many clients can participate in the sync. Does the .moneydancesync folder serve as a shared transaction journal that all clients can reference at will? If so, the only way the journal can ever be trimmed is when it can be determined that all clients have processed the oldest transactions. That would imply the need to have a client "registration" scheme, which would be difficult to implement passively. So maybe the .moneydancesync folder just never shrinks at all, and keeps getting bigger and bigger, forever. The reason for the two enormous .mdtxn files are that they are the initial status of the Primary dataset when I first configured sharing. Is that the way it works?
12 Posted by dtd on 06 Aug, 2023 08:22 PM
Regardless of how much Stuart is probably going to help you, I'm assuming you have backups, and maybe are even making extra backups at this point?
13 Posted by Stuart Beesley ... on 06 Aug, 2023 08:27 PM
Everything you are telling me says to do what I said.
Backup the good primary.
Disable sync.toolbox force reset sync settings.
Delete all the subfolders in .moneydancesync folder
Set up sync again.
(The large mdtxn files are dataset trunk files, but that won’t help you understand).
14 Posted by Milo C on 07 Aug, 2023 10:49 AM
Yes, @dtd, I do have lots of backups.
@Stuart and @dtd - I truly appreciate the experience and help you both offer here, and I will try your suggestions as usual, but I need to know: what am I doing wrong? I’ve been using MD for a long time and it has been a rock-solid accounting system for years.
My first data integrity issue happened a few years ago, when I naively put my synced dataset on iCloud (which is unsupported) and ended up with hundreds of duplicate transactions.
In the last few days, I’ve noticed that creating a new account hierarchy in a Secondary dataset seems to sync awkwardly to the Primary dataset. I ended up with some unfamiliar “Restored…” accounts and had to restart MD for the synced accounts to appear properly.
My datasets and sync config were recreated from scratch, just a few weeks ago. I notice exceptions popping up in the Console log from time to time and that it takes some time after transaction downloads before MD background threads finish up their activities. Is it bad to quit MD while all of this is going on?
I want to get back to the rock-solid MD days. What can I do differently?
15 Posted by Stuart Beesley ... on 07 Aug, 2023 12:19 PM
I still say set up sync from scratch again after using toolbox reset sync settings.
Do you use DB Connection or DB Folders?
What errors in console do you see?
16 Posted by dwg on 07 Aug, 2023 12:36 PM
FWIW Seeing a "Restored account ..." is really just a surfacing of an inconsistency in the data that may have been there for quite some time and just not visible, so it was probably not caused by any recent action, a recent action may have just lead to it surfacing.
These Restored accounts are generally caused by Moneydance encountering something that refers to a category or account that is no longer present. These references can be from a transaction, reminder, or sub-account. Because MD does not know the name of the category/account, it only has the ID number, it uses that to create the "Restored" accounts and uses this ID as part of the account name.
I do not know of any way of finding such issues in the data set, as far as I am aware you can only deal with it when it occurs.
17 Posted by Milo C on 07 Aug, 2023 04:45 PM
There were some stack traces but I didn't save those. Here is a warning that appears frequently:
warning: transaction is assigned to root account: [ParentTxn(cd82ec49-4417-4ba1-bc9e-fad557e2991a) 20230714 desc=2023 Contribution AIP; val=10417; stat= ; #splits=2; chk=; acct=My Finances; splits=SplitTxn: val=451730; amt=0; desc=2023 Contribution AIP; stat= ; cat=Restored 20d39add-2fe6-4a65-b918-17486ff1ffc8 from txn 20230714 2023 Contribution AIP 451730; tags=[(samt:451730)(acctid:20d39add-2fe6-4a65-b918-17486ff1ffc8)(invest.splittype:sec)(obj_type:)(id:4d20a16e-05ac-450a-9da9-4c976e7d712f)(pamt:0)(desc:2023 Contribution AIP)]; ], SplitTxn: val=-10417; amt=10417; desc=2023 Contribution AIP; stat= ; cat=; tags=[(samt:-10417)(acctid:2b10973b-eaad-4eff-83df-65117f0d15d9)(obj_type:)(invest.splittype:inc)(id:8efb3db6-b550-4ccb-a9af-9c0616ba136b)(pamt:10417)(desc:2023 Contribution AIP)]; ], ; ]
I have never had an account named "My Finances".
These "Contribution AIP" transactions appear in some of the "Restored" accounts.
18 Posted by Milo C on 07 Aug, 2023 04:48 PM
I also just noticed that MD is one of the apps that is currently marked as syncing to iCloud Drive. I don't recall setting that up. Should I uncheck that? Afraid what might happen if I do.
19 Posted by Stuart Beesley ... on 07 Aug, 2023 05:11 PM
What is your sync method? Anyway, I presume you mean in Mac/settings? This is fine. It doesn’t mean it’s being used. Where is your dataset actually located.
Txn assigned to root account is not good. Can you run toolbox. There is a fix for that.
20 Posted by Milo C on 07 Aug, 2023 06:01 PM
Sync method is "Dropbox Folders". Yes, in Mac iCloud settings. I will run toolbox to fix txn assigned to root account. My Master dataset is located ~/Library/Containers/Moneydance/Data/Documents.
When I open Toolbox, I get this:
Run 'FIX: Non-Hierarchical Security Acct Txns (& detect Orphans)'...
I've run that fix but it was only able to fix 1 of 4 txns. When I run it again, it can't fix any of the remaining 3. (I no longer have any Securities defined. I deleted the brokerage accounts from MD; they were not helpful.)
When I run "Detect and fix transactions assigned to 'root' account, I get this error:
ERROR - Cross-linked (or Orphaned) security tins detected.. Review Console. Run ‘FIX: Non-hierarchical Security Act Txns (& detect Orphans)’ >> no changes made
I uploaded the toolbox_output from the"FIX: Non-Hierarchical Security Acct Txns" run.
It looks like these two error situations are actually the same issue, and also related to the "Restored..." accounts I am seeing.
21 Posted by Stuart Beesley ... on 07 Aug, 2023 06:23 PM
😂 OK, perhaps I was over zealous on the check routines.. I have now removed the 'Non-hierarchical Security Act Txns' check from the 'Detect and fix transactions assigned to 'root' account' fix. Please can you:
backup
get / install latest PREVIEW toolbox from my site: https://yogi1967.github.io/MoneydancePythonScripts/
re-run 'Detect and fix transactions assigned to 'root' account'. routines. Does it work"?
then perhaps run the 'Non-hierarchical Security Act Txns' routine..
Let me know?
22 Posted by Milo C on 07 Aug, 2023 09:25 PM
So before I take any further action, I installed your preview Toolbox, ran
the Detect and fix transactions assigned to 'root' account routine. It
worked and popped up a big ledger window with 1 transaction in it:
[image: image.png]
I'm not sure what to do with this. I don't recognize the transaction, but I
suppose it must be "mine" somehow. Maybe it is a brokerage account
transaction that I haven't been following closely.
What should I do? Should I open an old archive copy of my dataset and look
for a transaction for $104.17?
Should I close this ledger window?
23 Posted by Milo C on 07 Aug, 2023 09:30 PM
Oh... I see, it created a new "TOOLBOX..." Investment account to hold this transaction. I closed the ledger window.
Will run the "Non-hierarchical Security Act Txns" routine next.
24 Posted by Milo C on 07 Aug, 2023 09:40 PM
So running the "Fix Non-hierarchical Security Act Txns" does complete but does not fix the 3 transactions.
The log (attached) says it can't fix the problem because the transaction is not in an Investment account. If these are leftover from my brokerage accounts, I really just want to make them disappear. What shouldI do?
25 Posted by Stuart Beesley ... on 07 Aug, 2023 09:44 PM
The simple solution is to find those 3 txns and delete and then recreate them. Possible?
26 Posted by Milo C on 07 Aug, 2023 09:57 PM
Possible... and done! I didn't recreate those txns because they look like they were from accounts I had deleted from MD, anyway.
Thank you.
So now, I am back to the place I was a couple of days ago.
I'm now working on my Master dataset, but afraid to start my Secondary dataset and work on that (on a different computer).
Is it bad to have multiple MD apps syncing together, all running at the same time?
(I don't think I had that situation going on, and I certainly wasn't going out of my way to do it.)
Should I go ahead and reset the sync? Stop sync on the Master, delete the .moneydancesync folder on Dropbox, then start setting up sync from scratch?
And how did I get into this situation in the first place? My sync setup was only a few weeks old.
27 Posted by Stuart Beesley ... on 07 Aug, 2023 10:09 PM
The non hierarchical txns were probably caused by using batch change on the ‘other’ side of the investment txns from a bank account side to change the acct/cat. This should never be done, but md allows it. It shouldn’t.
The txn assigned to root. Very odd. But probably caused by a sync screw up.
Multiple machines using sync open at once is fine.
Just do the sync force reset settings and set sync up again. On secondaries you will have to file/new/open sync to open the new sync copies. Make sure you pick the new sync and not old copies.
28 Posted by Milo C on 07 Aug, 2023 10:13 PM
If I’m resetting the sync and I am confident that my Master is sound, can I
just delete the entire Dropbox “.moneydancesync” folder?
I don’t recall ever entering any transactions that caused interactions
between an investment account and a bank account, but I could be wrong.
Anyway, I’ve removed the investment accounts so hopefully that won’t happen
again.
29 Posted by Stuart Beesley ... on 07 Aug, 2023 10:39 PM
I would just delete the sub folders.
30 Posted by Stuart Beesley ... on 07 Aug, 2023 10:40 PM
Well … misc Inc must go to a cat or acct. perhaps the missing cat/acct caused the issue.