syncing not working consistently

tgarson's Avatar

tgarson

13 Nov, 2023 06:17 AM

I don't know if this is going to be a recurrent problem of if it's transitory. I have made corrections to all entries that I believe are effected.
If the problem persists, you will hear from me again!
ISSUE:
I use the syncing feature with MD to keep three computers all "on the same page" in my heterogeneous network.
All systems are running the same version of MD.
2 computers run Opensuse Leap (15.4 and 15.5), one a laptop connecting via wifi and my workstation that is GB ethernet connected.
The third is a laptop operating via wifi running 64 bit Windows 7.
Syncing is via a shared folder on the local server.
Syncing mostly works but is not consistently completely updating from the server.
Entries made at my Linux workstation have begun to not show up at the laptops but if I enter a transaction at a laptop it does show up on my workstation.
If I edit an entry made on the workstation that did not appear on a laptop, it subsequently appears correctly on the laptop or shows up in a Recovered Account.

  1. 1 Posted by Stuart Beesley ... on 13 Nov, 2023 07:31 AM

    Stuart Beesley (Mr Toolbox)'s Avatar

    Grab help/console on each computer and upload that output here.

  2. 2 Posted by sth on 13 Nov, 2023 10:59 PM

    sth's Avatar

    How is the folder shared? Is it a NAS volume or some other sharing mechanism? I assume the the "server" with the shared folder is the "workstation" or is it another computer.

  3. 3 Posted by tgarson on 14 Nov, 2023 07:29 AM

    tgarson's Avatar

    My MD shared folder is on a dedicated Opensuse 15.4 server. It is shared
    via SAMBA so that my Win7 laptop can readily connect to it. All
    available updates to SMB networking are installed. All systems are 64bit.

    I checked "help" on the MD site: Found no answers there.

    My thought is there has to be some sort of handshake between the shared
    file on the server and a local file carried out by MD running on the app
    box. Perhaps something like a time stamp is kept in the local data file
    which is compared to one on the server file when the local MD is started
    up which might preclude MD from spending a lot of time verifying the
    data base. This "marker" might prevent MD from picking up an entry made
    from another system.

    I use a very old version of Multi-user Peachtree Complete in my
    business, which uses a B-Tree data structure and includes utilities for
    verifying and recovering from data inconsistencies. Peachtree uses a LOT
    of individual data files and prevents simultaneous R/W access to any set
    of them that are opened by a particular module. It is quite rare that I
    need to perform verification and even more rare that running recovery is
    in order. Peachtree is on a separate sub net of the same SAMBA network,
    also mostly Linux based.

    Is there any MD utility that can be run locally which can do a record by
    record verification, with option for making correction, of the local
    file against the shared file? (and/or vise versa)

    Thomas Garson
    Aural Technology, Ashland, OR
    By my calculation, the dynamic range of the universe is roughly 679dB,
    which is approximately 225 bits, collected at a rate 1.714287514x10^23 sps.

  4. 4 Posted by tgarson on 20 Nov, 2023 03:34 AM

    tgarson's Avatar

    Update:
    I found the source of at least part of my difficulties.
    My primary workstation runs Linux (Opensuse 15.4).
    If I forget to login to the file server after a system reboot, starting and exiting from Moneydance creates a new sync file locally (on the workstation), creating the same directory tree that it WOULD have connected to on the server. MD now "thinks" it has written new entries to the server when in fact it has not! Then, when I do login to the server, which provides the proper directory tree to my workstation, MD will not find the file with the new entries, nor will my laptop, because they have not been made to the sync folder.

    This is an issue that needs to be resolved by IK. Fortunately it should be a relatively simple fix: If MD can't find its sync files where they should be, an error message to that effect needs to be generated on the MD status bar, rather than MD just creating new files. I suggest that on the Syncing Page, found under "File", an option could be made selectable that enables creation of a new sync file in the specified location. A choice of what source should be used for the initial files used in this creation (default, local or other server?) might be a desirable convenience.

  5. 5 Posted by dtd on 20 Nov, 2023 03:48 AM

    dtd's Avatar

    If I forget to login to the file server after a system reboot,

    why does this have to be resolved by IK? Login! Don't make IK responsible for something you don't do, or forget to do.

  6. 6 Posted by dwg on 20 Nov, 2023 03:56 AM

    dwg's Avatar

    Your description makes it sound like you are storing your data files on the server. If this is the case Moneydance is not designed to work this way. It is intended to work with local files, when syncing is involved it compares the files held locally with those in a shared location on some sort of server, if the server is not available files are held pending the server coming online.

  7. 7 Posted by tgarson on 20 Nov, 2023 06:40 AM

    tgarson's Avatar

    DWG,

    You are partially correct, but far from entirely.
    Moneydance, by their own specifications, is designed to work with a
    supported server. Period. That is the entire point of having a sync
    directory. I am definitely storing files on a local server which is
    defined as an appropriate operation.

    I have found that Moneydance does write to a local file, subsequently
    updating the sync file. The problem with MD that I have discovered is
    that if it is unable to open the previously defined sync file, it
    automatically writes to a new sync file in a local directory structure
    which it creates, mimicking the servers directory structure. Thus,
    Moneydance ASSUMES that it has updated the sync directory on the server,
    when it actually has not.

    THIS IS A BUG.

    Its not a horrible one, especially once one learns what is happening and
    makes sure to not allow this situation to occur but MD should absolutely
    not work this way. I am very familiar with many of the gotchas that are
    involved with multiuser/multiportal networked software. In my previous
    post, I outlined that needed steps for IK to take to alleviate the issue.

    I will offer this:
    You are probably exclusively a Windows user. (If you're a Mac user, I
    apologize. However Apple is also notorious for trying to make the use of
    their computers "stupid simple" for the average user so this may apply
    to Apple as well.) Windows does quite a bit of the network housekeeping
    under the covers, which is both a bane and a boon. Once you have
    established a connection to a server from a Windows box, it will
    subsequently automatically re-log you on to that server without even
    asking because it retains your supposedly secure credentials. This is a
    horrible security risk, but Microsoft knows its customers and wants to
    make using your computer as easy as opening up the fridge for food or
    drink. The problem with this is you have to be physically present to
    open the fridge door. Hackers don't need to be present at your
    "terminal" to open things because of the many similarly inspired Windows
    security holes.

    I want software that does the task I need it to do without treating me
    like a 3 year old, at least when security is concerned. If I make a
    mistake and try to open Moneydance without being properly logged on to
    the server, I want it to tell me, not try to out think me that maybe I
    didn't REALLY want to connect to a server, or assume that the logon
    process has been completed without checking first. This is a fundamental
    rule of having file syncing and/or sharing and MD violates it.

    TG

    PS
    Most of the worlds networking and nearly all of the most powerful
    computing systems (even a lot of IBM stuff, even though they have their
    own in-house software base) run on Linux or Unix based software, not
    Windows. Macs are based on Unix, of which Linux is a variant.

  8. 8 Posted by tgarson on 20 Nov, 2023 06:53 AM

    tgarson's Avatar

    Because its good software design practice.

    The fix would be a trivial piece of code to write and would cover that base.

    I don't claim to be perfect (not even close). I'm busy and often
    distracted by circumstances. If you never forget to do anything, may I
    come worship at your feet?

    My 30+ year old DOS business accounting software requires me to log in
    when I open it and does a handshake with the server to make sure my
    login is correct AND that all necessary files have been properly opened
    for whatever access is needed to perform its correct use. It also
    encrypts my data, using a combination of the password and serial number
    for the key. Any IT manager worth their salt understands these needs.

    To steal an old but applicable quote: "This ain't Rocket Science".

    TG

  9. 9 Posted by sth on 20 Nov, 2023 09:52 PM

    sth's Avatar

    @tgarson, I understand the problem. I am not convinced that it is an IK problem. Most shared folders are just that, shared and always available at the location they are mounted. The fstab should mount the server at boot and periodically try to reconnect if you are trying to rely on OS shared folders.

    I am (primarily) a Mac user and this is how much of the Mac/linux software I use works. For security the user (and MD) should not actually be writing to the mount directory anyway and the mount should be through a proxy root privilege to assure mounting is secure and done with authorization.

    The Mac has this security such that the /Volumes folder where shared mounts are made is root:wheel access only. MD can not make a folder there unless a shared sub directory is mounted with the correct privilege.

    In terms of security, it is rocket science. And I actually have experience with rocket science and TBH it isn't that hard. I am fairly sure that your 30 year old DOS software is not cross platform and trying to make sense of a variety of OS and file systems.

  10. 10 Posted by tgarson on 21 Nov, 2023 05:36 AM

    tgarson's Avatar

    "Most" Linux software does not work "out of the box" as you suggest.
    Perhaps this is the case in the Mac world, which tries hard to be even
    more user friendly then Windows.

    As a security policy, I do not, and do not intend to, auto-mount server
    SMB (SAMBA) shares to any workstation. I use SMB4K to mount designated
    shares to a specific directory in my home folder. A password is required
    to mount each share. Passwords to server shares are NOT retained in
    fstab, nor in SMB4Ks setup file.

    SMB4K places a mounted share in the directory
    /home/username/smb4k/sharename subsequent to share login.
    Moneydance synced file directory (very_long_number)is accessed on my
    workstation via that mount as:
    /home/username/smb4k/server/sharename/Moneydance_Sync/very_long_number
    There is NO content in /home/username/smb4k/sharename until the share
    has been attached.

    If I neglect, or forget, to login to the appropriate server share and
    start Moneydance, on purpose or accidentally, moneydance automatically
    creates a new sync folder as
    /home/username/smb4k/server/sharename/Moneydance_Sync/new_very_long_filename
    and populates it with data copied from its regular local file structure.

    My contention is that Moneydance should not create this new structure
    silently. At the very least, Moneydance should notify you that it could
    not locate the specified resource and prompt for permission to create a
    new shared folder for synced data. This would be a telltale that
    something is amiss.

    As things now work, any subsequent startup of Moneydance by a user on
    any other computer who logs on to that server will load the synced files
    from that server which will not contain data entered via an instance of
    Moneydance when its platform was not logged in to that server.

    It is this automatic and unannounced creation of a "substitute" sync
    folder that violates good practice.

    Prior to the sync feature being added (2017?), an improvement that I
    wholeheartedly endorse, this was how Moneydance worked: If Moneydance
    could not find its data files, it would bark at you and refuse to start
    unless new files were created. I expect this would still the case if I
    started a new install of Moneydance. My pre 2017 version of Moneydance
    lived on my workstation but the data files were kept, even then, on my
    server. If for some reason those files were not accessible, Moneydance
    would tell me that and ask for guidance.

    TG

  11. Support Staff 11 Posted by Sean Reilly on 21 Nov, 2023 09:47 AM

    Sean Reilly's Avatar

    Hi TG,

    Thanks for posting this problem. It is something that I'll address in moneydance because it is automatically creating the sync folder if it doesn't exist. I made a change recently that caused this new behaviour because in some situations if a sync folder didn't already exist then moneydance would silently fail to sync all of your data. The "silently" part can be addressed by, as you say, showing a notification with a prompt to either create the missing sync folder or not sync until it reappears.

    I wouldn't say that moneydance is responsible for knowing anything about how your folder is shared - whether it's via a remote folder or something that is synced dropbox-style. I wonder if you've considered using a shared folder system such as owncloud or resilio sync? Something like that could more intelligently manage a shared folder without having to actually be connected to use it.

    Thanks,
    Sean

    --
    Sean Reilly
    Developer, The Infinite Kind
    https://infinitekind.com

  12. 12 Posted by sprimost on 21 Nov, 2023 03:39 PM

    sprimost's Avatar

    Fr: sprimost

    Sean, the issue is a bit bigger than an error on an unmounted file
    share. Even if one uses owncloud, etc., an error _accessing_ the shared
    disk space should report an error/notricifation.and not try to make an
    assumption. The latter shared solutions may not have a problem with
    availability except if you fill the area or don't pay your bill (!!).

  13. 13 Posted by Stuart Beesley ... on 21 Nov, 2023 03:48 PM

    Stuart Beesley (Mr Toolbox)'s Avatar

    I disagree with you here. If you are using iCloud or Dropbox sync then the folder is available or not. If the commands to make the structure work, then how on earth is MD supposed to know something is wrong (for iCloud and Dropbox)?

  14. 14 Posted by sprimost on 21 Nov, 2023 06:45 PM

    sprimost's Avatar

    Fr: sprimost

    Hmmm...perhaps I am missing something here, but in my (old) programming
    days, a "file write" returns an error code. The infomratin previously
    provided indicated that the error code caused the file to be rewritten
    to a location on the local disk. I think the request is that there
    should be _no_ assumption that a file is wriiten and that error codes
    mean action that is responsive to the user of the application.

  15. 15 Posted by Stuart Beesley ... on 21 Nov, 2023 07:42 PM

    Stuart Beesley (Mr Toolbox)'s Avatar

    The point is that there is no error code. The Java .mkdirs() method to create the folder structure is working / not causing an Exception… As Sean says, it needs to be tackled in a more ‘intelligent’ way.

  16. 16 Posted by sprimost on 22 Nov, 2023 12:53 AM

    sprimost's Avatar

    Fr: sprimost

    Now, I am confused because this does not seem logical. The observed
    behavior was that the shared device directory/folder was not available
    (due to it not being "mounted"), so MD wrote the transaction file (and
    the folder, if not yet created) in the local disk.
      (1) How did MD [sync?] know to that a write could not be completed on
    the shared device? Apparently it did.
      (2) How would it know that a folder did not exist on the local device
    (there could be many ways it knows where to write/create the folder).
    That is a separate question and another time because in this example,
    the create folder was created by the fact that $HOME is mounted as part
    of the startup
      (3) So, say, the method java.mkdirs() did fail, and as you said,
    there is no error code return. The the write to that "supposed" folder
    would fail. So, it a convoluted way, one would get an error later on,
    but that is a serious problem for any operation by MD.
      (4) Apparently, something is working with the method that had MD
    "know" it could not complete the write to the shared device, and then
    did used the method to create another folder on the $HOME folder. I
    would assume that the java.mkdirs() method was not involved in the
    initial write of the transactions, but MD did detect a failure at that
    time. So it seems to this old-guy that it is a logic problem with the
    error handling there-after.

    For help for me understanding "tgarson" issue, I have copied in-line his
    observation:

    SMB4K places a mounted share in the directory
    /home/username/smb4k/sharename subsequent to share login.
    Moneydance synced file directory (very_long_number)is accessed on my
    workstation via that mount as:
    /home/username/smb4k/server/sharename/Moneydance_Sync/very_long_number
    There is NO content in /home/username/smb4k/sharename until the share
    has been attached.

    If I neglect, or forget, to login to the appropriate server share and
    start Moneydance, on purpose or accidentally, moneydance automatically
    creates a new sync folder as
    /home/username/smb4k/server/sharename/Moneydance_Sync/new_very_long_filename

    and populates it with data copied from its regular local file structure.

    My contention is that Moneydance should not create this new structure
    silently.
    /scp

    On 11/21/23 14:42, Stuart Beesley (Mr Toolbox) wrote:

  17. 17 Posted by Stuart Beesley ... on 22 Nov, 2023 07:33 AM

    Stuart Beesley (Mr Toolbox)'s Avatar

    The MD code does not say “if sync folder not available then write somewhere else”! It says “write the folder, and write the files to the sync folder”. The Os is not returning an error. The Os is taking the folder create and file write commands and redirecting them somewhere (that you don’t want/like). That’s the point. There is no error. MD is happily writing files, and the Os is happily shoving them somewhere.

  18. System closed this discussion on 21 Feb, 2024 07:40 AM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac