Home → General Questions →

MD Data: File copy speed (slooooowwww)

• Edit

netr@cer

Aug 29, 2023 @ 01:59 PM

Any thoughts why a finder copy of an MD data file on a Mac takes forever? I am literally a watching KB size increase. Doesn't matter what mac I am using (M1 or Intel) or what medium (local drive to USB, local drive to network).

MD data file is only 86Mb. I can transfer a 3gb file to the network faster than my MD file gets there ....

Just looking to understand if this is expected behavior, or if it may be indicative of an underlying issue with my file.

Thanks

1. 1 Posted by Stuart Beesley ... on Aug 29, 2023 @ 02:25 PM

   

   If using Finder then a copy is a copy.. Where is it located?

   • Edit

2. 2 Posted by netr@cer on Aug 29, 2023 @ 03:53 PM

   

   In the documents folder. lt's weird though, it gets faster the deeper into the file. So the first couple MB count up at KB/s then it eventually increases and the last half of the file goes in a blink (as it should).

   I don't have iCloud syncing or anything like that either.

   • Edit

3. 3 Posted by Stuart Beesley ... on Aug 29, 2023 @ 04:39 PM

   

   Install and run toolbox. Click diagnose dataset and objects. Upload output here.

   • Edit

4. Support Staff 4 Posted by Maddy on Sep 01, 2023 @ 02:35 PM

   

   Hi,
   We are sorry to hear about the problem you have encountered.

   Delays in opening or closing/backing up your data file are usually due to either your data file or the backup location from the preferences being located in a shared folder.

   Could you try changing your backup location in the Moneydance preferences to the default location?

   You can do this by opening the Moneydance 'Preferences', selecting the Backup tab and un-checking the box for the backup location and restart Moneydance.

   If the problem persists, please send us your Moneydance error logs. The logs should include information about what might be causing it.

   They are saved on your computer as an errlog.txt file.

   You can get to your error logs as follow:

   1) navigate to Help->Console Window to show the console log in a window

   or

   2) in the Finder, press ⌘⇧G and then copy and paste ~/Library/Containers/com.infinitekind.MoneydanceOSX/Data/Library/Application Support into the field and press enter/return.

   At that point the finder will show an internal folder containing the errlog.txt file which includes the log.

   Finally you can paste the full content to your next reply.
   To add an attachment to this thread, please click on the "Browse" button.

   Thank you

   (I set your discussion to 'private' for the purpose).

   --
   Maddy, Infinite Kind Support

   • Edit

5. 5 Posted by netr@cer on Sep 29, 2023 @ 03:41 PM

   

   Sorry for the delay in replying. I went on vacation and then my pea brain forgot all about this until I tried today to make a backup copy of my data file to my NAS device.... :)
   Today's copy took about 20 minutes to reach 10ish Mb out of 88Mb, and then finished the last 78 in the blink of an eye.

   Output from Analyse Dataset Objs, Size & Files below.

   DATASET FILE ANALYSIS
    ====================

   Dataset path: /Users/name/Documents/MoneyDance/name.moneydance

   Dataset size: 88.1MB
   - settings file size: 101.3KB
   - key file size: 0.1KB
   - tiksync folder size: 82.1MB (with 1768 files)
     (note trunk file size: 72.4MB)
     (WAITING Sync 'Out' size: 9.2MB with 1766 files)
   - attachments size: 0.0MB (in 0 attachments)
   - archive size: 5.4MB (in 6 files)
   ---------------------------------------------
   Valid files size: 87.6MB (in 1776 files)

   Non-core file(s) size: 0.4MB (in 8 files)
      - Non-core: 0.1MB /Users/name/Documents/MoneyDance/name.moneydance/settings-20230410-1540_$SAVED$
      - Non-core: 0.0MB /Users/name/Documents/MoneyDance/name.moneydance/toolbox_update_log.txt
      - Non-core: 0.1MB /Users/name/Documents/MoneyDance/name.moneydance/settings-20230203-1532_$SAVED$
      - Non-core: 0.1MB /Users/name/Documents/MoneyDance/name.moneydance/settings-20230208-2218_$SAVED$
      - Non-core: 0.1MB /Users/name/Documents/MoneyDance/name.moneydance/settings-20230312-1148_$SAVED$
      - Non-core: 0.1MB /Users/name/Documents/MoneyDance/name.moneydance/settings-20230203-1612_$SAVED$
      - Non-core: 0.0MB /Users/name/Documents/MoneyDance/name.moneydance/securityquoteload.bpam2
      - Non-core: 0.0MB /Users/name/Documents/MoneyDance/name.moneydance/MRBPreferences.dict2

   LARGE (core) file(s) > 0.5MB....:
      - 0.9MB Mod: 2023-05-01 /Users/name/Documents/MoneyDance/name.moneydance/safe/archive/202303.mdtxnarchive
      - 1.3MB Mod: 2023-06-05 /Users/name/Documents/MoneyDance/name.moneydance/safe/archive/202304.mdtxnarchive
      - 0.8MB Mod: 2023-07-04 /Users/name/Documents/MoneyDance/name.moneydance/safe/archive/202305.mdtxnarchive
      - 0.8MB Mod: 2023-04-03 /Users/name/Documents/MoneyDance/name.moneydance/safe/archive/202302.mdtxnarchive
      - 0.9MB Mod: 2023-09-02 /Users/name/Documents/MoneyDance/name.moneydance/safe/archive/202307.mdtxnarchive
      - 0.8MB Mod: 2023-08-02 /Users/name/Documents/MoneyDance/name.moneydance/safe/archive/202306.mdtxnarchive
      - 72.4MB Mod: 2023-09-29 /Users/name/Documents/MoneyDance/name.moneydance/safe/tiksync/trunk

   DATABASE OBJECT COUNT (count) (est.size KBs):
   ------------------------------------------------------
   Object: mem_rpt 1 0.6
   Object: csnap 158903 35806.5
   Object: reminder 6 10.5
   Object: secsubtypes 1 0.2
   Object: olpmts 1 1.5 (containing 4 Online Payments)
   Object: txn 54964 35252.8
   Object: olsvc 8 25.9
   Object: csplit 19 3.3
   Object: oltxns 196 19.2
   Object: olpayees 1 2.1 (containing 7 Online Payees)
   Object: curr 242 70.1
   Object: acct 706 234.9
   Object: misc 3 7.2
    ==========
    TOTAL: 215051 71434.7

   QUICK SEARCH FOR OTHER DATASETS:
   ---------------------------------

   BACKUP FILES
   -------------
   Archive: Mod: 2023-09-22 /Users/name/Documents/MoneyDance/Backups/name.2023-09-22.moneydancearchive
   Archive: Mod: 2023-09-24 /Users/name/Documents/MoneyDance/Backups/name.2023-09-24.moneydancearchive
   Archive: Mod: 2023-09-26 /Users/name/Documents/MoneyDance/Backups/name.2023-09-26.moneydancearchive
   Archive: Mod: 2023-09-27 /Users/name/Documents/MoneyDance/Backups/name.2023-09-27.moneydancearchive
   Archive: Mod: 2023-09-28 /Users/name/Documents/MoneyDance/Backups/name.2023-09-28.moneydancearchive

   SYNC FOLDERS FOUND:
   ---------------------
   <NONE FOUND>

   (for a more extensive search please use Toolbox - Find my Datasets and Backups button

   <END>

   • Edit

6. Support Staff 6 Posted by Maddy on Oct 02, 2023 @ 12:16 PM

   

   Which version and build of Moneydance are you using?
   To check, open Moneydance and select Moneydance --> About Moneydance

   Please can you send us your Moneydance error logs. The logs should include information about what might be causing the problem.

   They are saved on your computer as an errlog.txt file.

   You can get to your error logs as follow:

   1) navigate to Help->Console Window to show the console log in a window

   or

   2) in the Finder, press ⌘⇧G and then copy and paste ~/Library/Containers/com.infinitekind.MoneydanceOSX/Data/Library/Application Support into the field and press enter/return.

   At that point the finder will show an internal folder containing the errlog.txt file which includes the log.

   Finally you can paste the full content to your next reply.
   To add an attachment to this thread, please click on the "Browse" button.

   Thank you

   --
   Maddy, Infinite Kind Support

   • Edit

7. 7 Posted by netr@cer on Oct 06, 2023 @ 04:12 PM

   

   v 2023.1 (5006).
   To be clear there are no errors or performance issues within the Moneydance application. The issue occurs when copying my MD datafile using finder. MD loads fine, backs up and exits fine, and transaction entry is quick.

   console log below with data anonymized....

   Starting Moneydance 2023.1 (5006) at Fri Oct 06 12:02:49 EDT 2023
   Settings Folder: /Users/username/Library/Containers/com.infinitekind.MoneydanceOSX/Data/Library/Application Support/Moneydance
   OS: Mac OS X 14.0 x86_64
   Architecture: intel64
   Java: 18.0.1 (Eclipse Adoptium)
   Is in sandbox: true; restricted file access: false
   Library path: /Applications/Moneydance.app/Contents/MacOS:/Applications/Moneydance.app/Contents/PlugIns/vm.jdk/Contents/Home/lib:/Applications/Moneydance.app/Contents/MacOS:/Applications/Moneydance.app/Contents/PlugIns/vm.jdk/Contents/Home/Frameworks
   initialized extension moneyforesight in 0.209 seconds
   python.path: /Users/username/Library/Containers/com.infinitekind.MoneydanceOSX/Data/Library/Application Support/Moneydance/python
   python.cachedir: /Users/username/Library/Containers/com.infinitekind.MoneydanceOSX/Data/Library/Application Support/Moneydance/python/cache
   python.cachedir.skip: false
   toolbox:2023/10/06-12:02:51:
   #############################################################################################################################
   toolbox: toolbox_init.py initializer script running - performing some quick checks, logging diagnostics, then will exit....
   #############################################################################################################################

   initialized extension toolbox in 0.244 seconds
   Font initialisation attempted from non-UI thread. Hopping over to dispatch thread...
   Initializing fonts:
     default: [[SF Pro Display, SF Display, Helvetica Neue, Helvetica, Lucida Grande, Dialog]]
     printing: [[SF Pro Display, SF Display, Helvetica Neue, Helvetica, Lucida Grande, Dialog]]
     monospaced: [[Gill Sans, Menlo, Monaco, Monospaced]]
     coding: [[Hack, Monospaced]]
   fonts resolved:
    default: SF Pro Display:13.0:style0
    mono: Gill Sans:13.0:style0
    coding: Hack:13.0:style0
    print: SF Pro Display:12.0:style0
    header: SF Pro Display:15.0:style0
    register: SF Pro Display:12.0:style0
    reportTitle: SF Pro Display:32.0:style0
    calendarTitle: SF Pro Display:12.0:style0
    reportHeader: SF Pro Display:11.0:style0
    detailTitle: SF Pro Display:15.0:style0
   Setting look and feel class to com.apple.laf.AquaLookAndFeel
   Font initialisation attempted from non-UI thread. Hopping over to dispatch thread...
   initialized extension yahooqt in 0.467 seconds
   opening last file: /Users/username/Documents/MoneyDance/filename.moneydance
   loading dataset: /Users/username/Documents/MoneyDance/filename.moneydance
   loading with 128 bit encryption key
   loading with 128 bit encryption key
   reading trunk data tiksync/trunk
   trunk data loaded (2.359 seconds), checking for incremental updates
   loading un-synced txn files...
    reading txn file 20231006145621_974.txn
    reading txn file 20231006151707_279.txn
    reading txn file 20231006151703_674.txn
    reading txn file 20231006153243_149.txn
    reading txn file 20231006151446_731.txn
    reading txn file 20231006144641_598.txn
    reading txn file 20231006151349_075.txn
    reading txn file 20231006150958_903.txn
    reading txn file 20231006155622_438.txn
    reading txn file 20231006153152_552.txn
    reading txn file 20231006144635_478.txn
    reading txn file 20231006151042_183.txn
    reading txn file 20231006153240_573.txn
    reading txn file 20231006154400_094.txn
    reading txn file 20231006155754_151.txn
    reading txn file 20231006155900_014.txn
    reading txn file 20231006151039_451.txn
    reading txn file 20231006151615_351.txn
    reading txn file 20231006145637_423.txn
    reading txn file 20231006151144_914.txn
    reading txn file 20231006155412_686.txn
    reading txn file 20231006153227_011.txn
    reading txn file 20231006151422_647.txn
    reading txn file 20231006151459_320.txn
    reading txn file 20231006154324_835.txn
    reading txn file 20231006151240_354.txn
    reading txn file 20231006154515_681.txn
    reading txn file 20231006153437_302.txn
    reading txn file 20231006144549_010.txn
    reading txn file 20231006154347_827.txn
    reading txn file 20231006154413_896.txn
    reading txn file 20231006151002_970.txn
    reading txn file 20231006155824_678.txn
    reading txn file 20231006155808_594.txn
   loading synced txn files...
   replacing root: null () with new root: filename (10611bc6-6492-4b97-b70d-cc6ca59d1e58
   toolbox:2023/10/06-12:03:01:
   -----------------------------------------------------
   Toolbox - quick information:
   -----
   MD CONSOLE FILE LOCATION: '/Users/username/Library/Containers/com.infinitekind.MoneydanceOSX/Data/Library/Application Support/Moneydance/errlog.txt'
   MD CONFIG/PREFERENCES LOCATION: '/Users/username/Library/Containers/com.infinitekind.MoneydanceOSX/Data/Library/Application Support/Moneydance/config.dict'
   ... Mac Finder path for above: '/Users/username/Library/Containers/Moneydance/Data/Library/Application Support/Moneydance'
   -----
   BACKUPS - Save Daily: True
   BACKUPS - Keep no more than: 5 backups
   BACKUPS - Separate Backup Foldr: True
   BACKUPS - Backup Folder: '/Users/username/Documents/MoneyDance/Backups' (backup location exists)
   ..key - 'backup.location': '/Users/username/Documents/MoneyDance/Backups'
   ..key - 'backup.last_browsed': '<not set>'
   ..key - 'backup.last_saved': '<not set>'
   ..key - '_default_backup_dir': '<not set>'
   -----
   System Default Locale Cty/Lang: 'US' / 'en'
   MD Preference Locale Ctry/Lang: 'US' / 'en'
   MD Locale Cty/Lang: 'US' / 'en'
   Moneydance decimal point: '.'
   -----
   MD EXECUTION MODE: 2 (Normal)
   JVM - Available processor cores: 16
   JVM - Maximum memory possible: 57.6 GB
   JVM - Total memory allocated: 1.6 GB (used 1.0 GB / free 0.6 GB)
   -----------------------------------------------------

   ConnectionTask for Update_If_Needed done, result true
   Update info received with successful signature verification
   Version info retrieved; we are up to date!
   Ran 0 of 2 MDP tasks @ Fri Oct 06 12:03:09 EDT 2023) forcedRefresh=true
   Ran 0 of 2 MDP tasks @ Fri Oct 06 12:03:11 EDT 2023) forcedRefresh=true

   • Edit

8. 8 Posted by dtd on Oct 06, 2023 @ 11:46 PM

   

   This may be an issue with MD 5006 and Sonoma. MD is not currently verified to work properly with Sonoma, as new Mac OS's break things with software, and each software company (not only MD) has to catch up at some point.

   In other threads, I've seen reports of problems with icloud sync as well as simply copying files through MD. The current alpha testing has found problems with MD and Sonoma and fixes are being made - but again, still alpha, though beta(Preview) should be out fairly soon.

   Still, that will be a beta, and all that implies.

   • Edit

9. 9 Posted by netr@cer on Oct 10, 2023 @ 04:46 PM

   

   This issue was occurring months ago while I was on Monterey so Sonoma is not the root cause, but Sonoma does still have the issue.

   I did a dataset shrink using toolbox and the performance is much better (but still not behaving as it should).

   Based on the toolbox analyze log file data, I suspect that MacOS is having fits with the internal file structure of the moneydance file and it doesn't like dealing with the thousands of little files inside the data file.

   From what I can tell (in both before and after tests), the trunk file (being a single large file) copies along just fine, but the tiksync/out folder (comprising thousands of tiny files) is thrashing the OS with file open and close commands.

   I accept that on most desktop OS's copying 100mb as a single file is super fast while copying 100Mb made up of 1,000 10K files will take longer, but this does seem a bit excessive with today's compute power.

   I'm not sure if something else is at play.
   I suspect I'm in the minority in that I manually backup my primary moneydance file to a NAS and this might be why no one else has reported this issue yet. Interestingly, copying moneydancearhive files is instantaneous and does not experience any performance problems.

   I'm hoping something can be done so I don't have to keep shrinking my dataset (otherwise copy performance will continue to degrade over time).

   • Edit

10. 10 Posted by Stuart Beesley ... on Oct 10, 2023 @ 05:20 PM

    

    As you are using Os features to copy your dataset, then the issue is with MacOS and not MD. I would suggest you have an issue with your hard disk or network drive. Perhaps there are cache/buffering settings. Dunno……

    Why do you copy it to your network? What happens if you export a backup and then copy that to the network?

    • Edit

11. 11 Posted by netr@cer on Oct 10, 2023 @ 05:35 PM

    

    Yes, this has always been an os copy issue. I copy the file so I have my actual working file, this way I don't need to perform a "restore process" from a generated backup. Over the years I have found that backups aren't quite the same as a raw copy of a dataset, whether that's sql, quicken, or in this case MD.

    The autogenerated backup files copy just fine to the network.
    It's not my laptop because I've tried it on three difference Mac laptops (Intel and Apple Silicon) and the problem exists on all of them. It's not the NAS, because it happens when I copy to a USB stick as well (though I suppose I could have a bad usb stick and nas device; but what are the odds.)

    Why copy to the network? I have a multi-tiered backup strategy, I copy all my important data from all the computers in the house to my NAS, and my NAS syncs to alternating USB drives plugged into the NAS and I rotate them into a bank safe at the end of the month. I always have a copy offsite and always a copy handy incase of a laptop crash.

    I currently don't do cloud storage, though that might change now that apple has advance encryption features on iCloud (my encryption key, not theirs). Unfortunately I still have one or two devices that can't upgrade to the latest OS, so I can't enable AE until those devices are off my iCloud account.

    • Edit

12. 12 Posted by Stuart Beesley ... on Oct 10, 2023 @ 05:50 PM

    

    Well, the backup (.moneydancearchive) is an exact copy. It’s basically a recursive zip of the folder. That’s it.

    • Edit

13. 13 Posted by netr@cer on Oct 10, 2023 @ 06:20 PM

    

    Hmm, look at that. I couldn't open it with show package contents, didn't think to try opening with a zip tool.

    I guess I can change my habits on this and just backup the backup file....

    Still doesn't explain the slowness, but if I can stop the headache by not hitting my head against the wall, so be it :D

    • Edit

14. netr@cer closed this discussion on Oct 10, 2023 @ 06:21 PM.