Home → Syncing Moneydance with iPhone, iPod Touch, iPad, and Android Devices →

Dropbox sync is very, very slow

• Edit

Keith Beattie

17 Apr, 2024 06:43 AM

Hello,

I recently got a new mac laptop and migrated my Moneydance over to it, as I have done several times over the years. Sync'ing with Dropbox has (until now) made it quite easy.

Unfortunately now with nearly each transaction (new entry or editing an existing entry) Moneydance freezes for about 60 seconds. This does not happen when I have disabled sync and it doesn't always happen with sync enabled but does happen frequently enough to make it very frustrating. When running the MacOS activity monitor and showing network traffic I can see that when it does hang it appears to be sending a couple of GB over the network. When it finishes, Moneydance unfreezes.

I've disabled sync on all my previous installs, and disabled sync on the new laptop, then deleted the .moneydance folder in Dropbox, then re-enabled sync on the one new laptop. The problem doesn't happen when sync is disabled, but as soon as I re-enable it (which takes hours to complete) it starts happening again.

It appears that something with my datafile is clearly messed up, any way to rebuild it in a way that will get sync working again properly?

1. 1 Posted by Stuart Beesley ... on 17 Apr, 2024 07:07 AM

   

   Run toolbox extension. Click the analyse dataset and objects button. Upload the output here. Also upload your help/console file.

   • Edit

2. 2 Posted by Keith Beattie on 17 Apr, 2024 09:12 PM

   

   That appears to have some personal, or at least potentially compromisable personal info in it. Maybe we can make this a private discussion?

   Meanwhile, the help/console output looks like it is revealing the issue. I re-enabled sync and can see it looping with the following exception:

   20240417-13:55:31.118: Beattie[2 tasks] uploading buffer file to syncFolder
   uploading file v3/a88aafc2-d715-4b15-8a14-a26099c29f44.mdtxn to path /.moneydancesync/3d28dcef-4766-4e87-a786-34059e510b81/v3/a88aafc2-d715-4b15-8a14-a26099c29f44.mdtxn
   uploading file /Users/ksb/Library/Containers/com.infinitekind.MoneydanceOSX/Data/tmp/moneydance_db_sync4691809752647992328 to dropbox path v3/a88aafc2-d715-4b15-8a14-a26099c29f44.mdtxn
   20240417-13:56:37.639: Error uploading outgoing transactions: com.dropbox.core.v2.files.UploadErrorException: Exception in 2/files/upload: "payload_too_large"
   Error: Exception in 2/files/upload: "payload_too_large"
   com.dropbox.core.v2.files.UploadErrorException: Exception in 2/files/upload: "payload_too_large"
       at com.dropbox.core.v2.files.UploadUploader.newException(UploadUploader.java:37)
       at com.dropbox.core.v2.files.UploadUploader.newException(UploadUploader.java:23)
       at com.dropbox.core.DbxUploader.finish(DbxUploader.java:287)
       at com.dropbox.core.DbxUploader.uploadAndFinish(DbxUploader.java:126)
       at com.dropbox.core.DbxUploader.uploadAndFinish(DbxUploader.java:96)
       at com.dropbox.core.v2.DbxUploadStyleBuilder.uploadAndFinish(DbxUploadStyleBuilder.java:92)
       at com.moneydance.apps.md.controller.sync.DropboxAPISyncFolder.writeFile(DropboxAPISyncFolder.java:249)
       at com.moneydance.apps.md.controller.sync.DropboxAPISyncFolder.writeFile(DropboxAPISyncFolder.java:222)
       at com.moneydance.apps.md.controller.sync.EncryptedSyncFolder.writeFile(EncryptedSyncFolder.kt:71)
       at com.infinitekind.tiksync.Syncer.pushOutgoingTxns(Syncer.kt:1454)
       at com.infinitekind.tiksync.Syncer.access$pushOutgoingTxns(Syncer.kt:27)
       at com.infinitekind.tiksync.Syncer$MainSyncTask.performSyncTask(Syncer.kt:938)
       at com.infinitekind.tiksync.Syncer$SyncTask.run(Syncer.kt:731)
   

   The file that it is failing to upload is not the same each time. It's been doing this all morning long (now 5 hours) with 2mins between each loop/failure.

   Maybe one of your toolbox tricks will squeeze the size of those "too large" transactions? Or identify the specific transaction so I can edit it to make it smaller - like remove an attachment?

   Thanks,
   ksb

   • Edit

3. 3 Posted by Stuart Beesley ... on 17 Apr, 2024 09:29 PM

   

   Run toolbox extension. Click the analyse dataset and objects button. Upload that output here.

   • Edit

4. 4 Posted by Keith Beattie on 18 Apr, 2024 12:48 AM

   

   Here's the file.

   • toolbox_output.txt 17.7 KB
   • Edit

5. 5 Posted by Stuart Beesley ... on 18 Apr, 2024 06:26 AM

   

   I’m not surprised. You have a stuck / damaged massive file that just keeps trying to upload.

   Your dataset data is 30MB but your dataset is now 1GB+

   Is this your master dataset and do you consider it to be correct?

   Do you have other computers you sync to, or it is just mobile?

   I’m presuming only computer and only to mobile. If not, then stop here.

   BACKUP first. File / export backup.

   • use toolbox, update mode, advanced. Force turn off sync. Then force reset sync settings.
   • quit MD
   • using finder, or explorer, find and delete the 720mb+ file:

   /Users/ksb/Library/Containers/com.infinitekind.MoneydanceOSX/Data/Documents/Beattie.moneydance/safe/tiksync/uploadbuf
   

   Or launch terminal and type

   rm /Users/ksb/Library/Containers/com.infinitekind.MoneydanceOSX/Data/Documents/Beattie.moneydance/safe/tiksync/uploadbuf
   

   • Now launch MD, rerun toolbox. This time run shrink dataset. Say yes to purge out folder if asked and enter zero days. Let that run.

   • Restart MD.

   • Rerun the toolbox analyse dataset and objects and upload that here

   • Set up sync again from scratch and then do the same on your mobile device. You will have to disconnect and then reconnect to the new sync dataset. Make sure you do a test txn to ensure it syncs both ways ok.

   • report back. We may well have to also manually delete your old Dropbox sync data.

   • Edit

6. 6 Posted by Keith Beattie on 18 Apr, 2024 07:25 PM

   

   Thanks so much! After following your steps it all seems to be working again properly now. Attached is the dataset analysis done right after the db shrink (but before setting up sync'ing again).

   A couple of follow up Qs:

   Q1) Any idea how this happened as to avoid it in the future? Given that I've been using this (single file) for over 10 years, it's not really that pressing, but just in case you know.

   When setting up the sync on my phone, it asked which of two datasets it should sync with, including their last update timestamp. I picked the latest updated and changes are sync'd between the phone and my laptop now.

   Q2) When looking on dropbox, I see 3 dirs under the .moneydance dir, can any of those be removed now? If so, how do I identify which one(s) to remove?

   Q3) Any suggestions on how to set up another sync on a separate machine? As in, should I remove existing files there before doing that?

   Thanks again for you help and software.
   ksb

   • toolbox_output_dataset_analysis_post_db_shrink.txt 4.88 KB
   • Edit

7. 7 Posted by Stuart Beesley ... on 18 Apr, 2024 07:35 PM

   

   Your dataset is now looking great.. That's a 1.5GB size reduction to 41MB! 😀

   Q1) Nope. But is this a clue:

   I recently got a new mac laptop and migrated my Moneydance over to it, as I have done several times over the years. Sync'ing with Dropbox has (until now) made it quite easy.
   

   Could you have copied a file in an open state?

   Also, another person had exactly the same problem this week (also with 1.5GB of stuck data).... Hmmm... Dunno yet...

   Q2). Toolbox, main screen, will tell you something like this:
   

   Dataset internal UUID: 7d7682ea-d5ae-4f98-9101-5387ee81b36c
   

   This is the name of the .moneydancesync/xxxxx folder in Dropbox to KEEP.. You can delete the others....

   Q3) On the other machine, get dropbox working first, then install MD, then file/New/Open Sync'd file/ Dropbox. Select the dataset and it will magically appear... If you already had sync working before, then still do the file/new thing, but afterwards you will need to delete any old xxx.moneydance files...

   Toolbox 🧰 +1 😃

   • Edit

8. 8 Posted by Stuart Beesley ... on 18 Apr, 2024 08:21 PM

   

   Oh yes. I forgot. If on 2024 go to console. Bottom. Click (…) show sync folder also shows your current folder (to keep)

   • Edit

9. System closed this discussion on 18 Jul, 2024 08:30 PM.