syncing not working consistently
I don't know if this is going to be a recurrent problem of if it's transitory. I have made corrections to all entries that I believe are effected.
If the problem persists, you will hear from me again!
ISSUE:
I use the syncing feature with MD to keep three computers all "on the same page" in my heterogeneous network.
All systems are running the same version of MD.
2 computers run Opensuse Leap (15.4 and 15.5), one a laptop connecting via wifi and my workstation that is GB ethernet connected.
The third is a laptop operating via wifi running 64 bit Windows 7.
Syncing is via a shared folder on the local server.
Syncing mostly works but is not consistently completely updating from the server.
Entries made at my Linux workstation have begun to not show up at the laptops but if I enter a transaction at a laptop it does show up on my workstation.
If I edit an entry made on the workstation that did not appear on a laptop, it subsequently appears correctly on the laptop or shows up in a Recovered Account.
Keyboard shortcuts
Generic
| ? | Show this help |
|---|---|
| ESC | Blurs the current field |
Comment Form
| r | Focus the comment reply box |
|---|---|
| ^ + ↩ | Submit the comment |
You can use Command ⌘ instead of Control ^ on Mac
1 Posted by Stuart Beesley ... on 13 Nov, 2023 07:31 AM
Grab help/console on each computer and upload that output here.
2 Posted by sth on 13 Nov, 2023 10:59 PM
How is the folder shared? Is it a NAS volume or some other sharing mechanism? I assume the the "server" with the shared folder is the "workstation" or is it another computer.
3 Posted by tgarson on 14 Nov, 2023 07:29 AM
My MD shared folder is on a dedicated Opensuse 15.4 server. It is shared
via SAMBA so that my Win7 laptop can readily connect to it. All
available updates to SMB networking are installed. All systems are 64bit.
I checked "help" on the MD site: Found no answers there.
My thought is there has to be some sort of handshake between the shared
file on the server and a local file carried out by MD running on the app
box. Perhaps something like a time stamp is kept in the local data file
which is compared to one on the server file when the local MD is started
up which might preclude MD from spending a lot of time verifying the
data base. This "marker" might prevent MD from picking up an entry made
from another system.
I use a very old version of Multi-user Peachtree Complete in my
business, which uses a B-Tree data structure and includes utilities for
verifying and recovering from data inconsistencies. Peachtree uses a LOT
of individual data files and prevents simultaneous R/W access to any set
of them that are opened by a particular module. It is quite rare that I
need to perform verification and even more rare that running recovery is
in order. Peachtree is on a separate sub net of the same SAMBA network,
also mostly Linux based.
Is there any MD utility that can be run locally which can do a record by
record verification, with option for making correction, of the local
file against the shared file? (and/or vise versa)
Thomas Garson
Aural Technology, Ashland, OR
By my calculation, the dynamic range of the universe is roughly 679dB,
which is approximately 225 bits, collected at a rate 1.714287514x10^23 sps.
4 Posted by tgarson on 20 Nov, 2023 03:34 AM
Update:
I found the source of at least part of my difficulties.
My primary workstation runs Linux (Opensuse 15.4).
If I forget to login to the file server after a system reboot, starting and exiting from Moneydance creates a new sync file locally (on the workstation), creating the same directory tree that it WOULD have connected to on the server. MD now "thinks" it has written new entries to the server when in fact it has not! Then, when I do login to the server, which provides the proper directory tree to my workstation, MD will not find the file with the new entries, nor will my laptop, because they have not been made to the sync folder.
This is an issue that needs to be resolved by IK. Fortunately it should be a relatively simple fix: If MD can't find its sync files where they should be, an error message to that effect needs to be generated on the MD status bar, rather than MD just creating new files. I suggest that on the Syncing Page, found under "File", an option could be made selectable that enables creation of a new sync file in the specified location. A choice of what source should be used for the initial files used in this creation (default, local or other server?) might be a desirable convenience.
5 Posted by dtd on 20 Nov, 2023 03:48 AM
If I forget to login to the file server after a system reboot,
why does this have to be resolved by IK? Login! Don't make IK responsible for something you don't do, or forget to do.
6 Posted by dwg on 20 Nov, 2023 03:56 AM
Your description makes it sound like you are storing your data files on the server. If this is the case Moneydance is not designed to work this way. It is intended to work with local files, when syncing is involved it compares the files held locally with those in a shared location on some sort of server, if the server is not available files are held pending the server coming online.
7 Posted by tgarson on 20 Nov, 2023 06:40 AM
DWG,
You are partially correct, but far from entirely.
Moneydance, by their own specifications, is designed to work with a
supported server. Period. That is the entire point of having a sync
directory. I am definitely storing files on a local server which is
defined as an appropriate operation.
I have found that Moneydance does write to a local file, subsequently
updating the sync file. The problem with MD that I have discovered is
that if it is unable to open the previously defined sync file, it
automatically writes to a new sync file in a local directory structure
which it creates, mimicking the servers directory structure. Thus,
Moneydance ASSUMES that it has updated the sync directory on the server,
when it actually has not.
THIS IS A BUG.
Its not a horrible one, especially once one learns what is happening and
makes sure to not allow this situation to occur but MD should absolutely
not work this way. I am very familiar with many of the gotchas that are
involved with multiuser/multiportal networked software. In my previous
post, I outlined that needed steps for IK to take to alleviate the issue.
I will offer this:
You are probably exclusively a Windows user. (If you're a Mac user, I
apologize. However Apple is also notorious for trying to make the use of
their computers "stupid simple" for the average user so this may apply
to Apple as well.) Windows does quite a bit of the network housekeeping
under the covers, which is both a bane and a boon. Once you have
established a connection to a server from a Windows box, it will
subsequently automatically re-log you on to that server without even
asking because it retains your supposedly secure credentials. This is a
horrible security risk, but Microsoft knows its customers and wants to
make using your computer as easy as opening up the fridge for food or
drink. The problem with this is you have to be physically present to
open the fridge door. Hackers don't need to be present at your
"terminal" to open things because of the many similarly inspired Windows
security holes.
I want software that does the task I need it to do without treating me
like a 3 year old, at least when security is concerned. If I make a
mistake and try to open Moneydance without being properly logged on to
the server, I want it to tell me, not try to out think me that maybe I
didn't REALLY want to connect to a server, or assume that the logon
process has been completed without checking first. This is a fundamental
rule of having file syncing and/or sharing and MD violates it.
TG
PS
Most of the worlds networking and nearly all of the most powerful
computing systems (even a lot of IBM stuff, even though they have their
own in-house software base) run on Linux or Unix based software, not
Windows. Macs are based on Unix, of which Linux is a variant.
8 Posted by tgarson on 20 Nov, 2023 06:53 AM
Because its good software design practice.
The fix would be a trivial piece of code to write and would cover that base.
I don't claim to be perfect (not even close). I'm busy and often
distracted by circumstances. If you never forget to do anything, may I
come worship at your feet?
My 30+ year old DOS business accounting software requires me to log in
when I open it and does a handshake with the server to make sure my
login is correct AND that all necessary files have been properly opened
for whatever access is needed to perform its correct use. It also
encrypts my data, using a combination of the password and serial number
for the key. Any IT manager worth their salt understands these needs.
To steal an old but applicable quote: "This ain't Rocket Science".
TG
9 Posted by sth on 20 Nov, 2023 09:52 PM
@tgarson, I understand the problem. I am not convinced that it is an IK problem. Most shared folders are just that, shared and always available at the location they are mounted. The fstab should mount the server at boot and periodically try to reconnect if you are trying to rely on OS shared folders.
I am (primarily) a Mac user and this is how much of the Mac/linux software I use works. For security the user (and MD) should not actually be writing to the mount directory anyway and the mount should be through a proxy root privilege to assure mounting is secure and done with authorization.
The Mac has this security such that the /Volumes folder where shared mounts are made is root:wheel access only. MD can not make a folder there unless a shared sub directory is mounted with the correct privilege.
In terms of security, it is rocket science. And I actually have experience with rocket science and TBH it isn't that hard. I am fairly sure that your 30 year old DOS software is not cross platform and trying to make sense of a variety of OS and file systems.
10 Posted by tgarson on 21 Nov, 2023 05:36 AM
"Most" Linux software does not work "out of the box" as you suggest.
Perhaps this is the case in the Mac world, which tries hard to be even
more user friendly then Windows.
As a security policy, I do not, and do not intend to, auto-mount server
SMB (SAMBA) shares to any workstation. I use SMB4K to mount designated
shares to a specific directory in my home folder. A password is required
to mount each share. Passwords to server shares are NOT retained in
fstab, nor in SMB4Ks setup file.
SMB4K places a mounted share in the directory
/home/username/smb4k/sharename subsequent to share login.
Moneydance synced file directory (very_long_number)is accessed on my
workstation via that mount as:
/home/username/smb4k/server/sharename/Moneydance_Sync/very_long_number
There is NO content in /home/username/smb4k/sharename until the share
has been attached.
If I neglect, or forget, to login to the appropriate server share and
start Moneydance, on purpose or accidentally, moneydance automatically
creates a new sync folder as
/home/username/smb4k/server/sharename/Moneydance_Sync/new_very_long_filename
and populates it with data copied from its regular local file structure.
My contention is that Moneydance should not create this new structure
silently. At the very least, Moneydance should notify you that it could
not locate the specified resource and prompt for permission to create a
new shared folder for synced data. This would be a telltale that
something is amiss.
As things now work, any subsequent startup of Moneydance by a user on
any other computer who logs on to that server will load the synced files
from that server which will not contain data entered via an instance of
Moneydance when its platform was not logged in to that server.
It is this automatic and unannounced creation of a "substitute" sync
folder that violates good practice.
Prior to the sync feature being added (2017?), an improvement that I
wholeheartedly endorse, this was how Moneydance worked: If Moneydance
could not find its data files, it would bark at you and refuse to start
unless new files were created. I expect this would still the case if I
started a new install of Moneydance. My pre 2017 version of Moneydance
lived on my workstation but the data files were kept, even then, on my
server. If for some reason those files were not accessible, Moneydance
would tell me that and ask for guidance.
TG
Support Staff 11 Posted by Sean Reilly on 21 Nov, 2023 09:47 AM
Hi TG,
Thanks for posting this problem. It is something that I'll address in moneydance because it is automatically creating the sync folder if it doesn't exist. I made a change recently that caused this new behaviour because in some situations if a sync folder didn't already exist then moneydance would silently fail to sync all of your data. The "silently" part can be addressed by, as you say, showing a notification with a prompt to either create the missing sync folder or not sync until it reappears.
I wouldn't say that moneydance is responsible for knowing anything about how your folder is shared - whether it's via a remote folder or something that is synced dropbox-style. I wonder if you've considered using a shared folder system such as owncloud or resilio sync? Something like that could more intelligently manage a shared folder without having to actually be connected to use it.
Thanks,
Sean
--
Sean Reilly
Developer, The Infinite Kind
https://infinitekind.com
12 Posted by sprimost on 21 Nov, 2023 03:39 PM
Fr: sprimost
Sean, the issue is a bit bigger than an error on an unmounted file
share. Even if one uses owncloud, etc., an error _accessing_ the shared
disk space should report an error/notricifation.and not try to make an
assumption. The latter shared solutions may not have a problem with
availability except if you fill the area or don't pay your bill (!!).
13 Posted by Stuart Beesley ... on 21 Nov, 2023 03:48 PM
I disagree with you here. If you are using iCloud or Dropbox sync then the folder is available or not. If the commands to make the structure work, then how on earth is MD supposed to know something is wrong (for iCloud and Dropbox)?
14 Posted by sprimost on 21 Nov, 2023 06:45 PM
Fr: sprimost
Hmmm...perhaps I am missing something here, but in my (old) programming
days, a "file write" returns an error code. The infomratin previously
provided indicated that the error code caused the file to be rewritten
to a location on the local disk. I think the request is that there
should be _no_ assumption that a file is wriiten and that error codes
mean action that is responsive to the user of the application.
15 Posted by Stuart Beesley ... on 21 Nov, 2023 07:42 PM
The point is that there is no error code. The Java .mkdirs() method to create the folder structure is working / not causing an Exception… As Sean says, it needs to be tackled in a more ‘intelligent’ way.
16 Posted by sprimost on 22 Nov, 2023 12:53 AM
Fr: sprimost
Now, I am confused because this does not seem logical. The observed
behavior was that the shared device directory/folder was not available
(due to it not being "mounted"), so MD wrote the transaction file (and
the folder, if not yet created) in the local disk.
(1) How did MD [sync?] know to that a write could not be completed on
the shared device? Apparently it did.
(2) How would it know that a folder did not exist on the local device
(there could be many ways it knows where to write/create the folder).
That is a separate question and another time because in this example,
the create folder was created by the fact that $HOME is mounted as part
of the startup
(3) So, say, the method java.mkdirs() did fail, and as you said,
there is no error code return. The the write to that "supposed" folder
would fail. So, it a convoluted way, one would get an error later on,
but that is a serious problem for any operation by MD.
(4) Apparently, something is working with the method that had MD
"know" it could not complete the write to the shared device, and then
did used the method to create another folder on the $HOME folder. I
would assume that the java.mkdirs() method was not involved in the
initial write of the transactions, but MD did detect a failure at that
time. So it seems to this old-guy that it is a logic problem with the
error handling there-after.
For help for me understanding "tgarson" issue, I have copied in-line his
observation:
SMB4K places a mounted share in the directory
/home/username/smb4k/sharename subsequent to share login.
Moneydance synced file directory (very_long_number)is accessed on my
workstation via that mount as:
/home/username/smb4k/server/sharename/Moneydance_Sync/very_long_number
There is NO content in /home/username/smb4k/sharename until the share
has been attached.
If I neglect, or forget, to login to the appropriate server share and
start Moneydance, on purpose or accidentally, moneydance automatically
creates a new sync folder as
/home/username/smb4k/server/sharename/Moneydance_Sync/new_very_long_filename
and populates it with data copied from its regular local file structure.
My contention is that Moneydance should not create this new structure
silently.
/scp
On 11/21/23 14:42, Stuart Beesley (Mr Toolbox) wrote:
17 Posted by Stuart Beesley ... on 22 Nov, 2023 07:33 AM
The MD code does not say “if sync folder not available then write somewhere else”! It says “write the folder, and write the files to the sync folder”. The Os is not returning an error. The Os is taking the folder create and file write commands and redirecting them somewhere (that you don’t want/like). That’s the point. There is no error. MD is happily writing files, and the Os is happily shoving them somewhere.