importQIFIntoAccount question

rapscallion's Avatar

rapscallion

28 May, 2019 02:43 PM

I am attempting to load transactions parsed from a PDF file into Moneydance. I am using the importQIFIntoAccount Java API call which seems to work just fine when Common.QIF_MODE_TRANSFER is used, but no transactions appear when using Common.QIF_MODE_DOWNLOAD is set. I am not sure why, any ideas...? That is the only parameter I am changing on the API call, the file content I am using is not changing.

I am tracing this using Eclipse and can see that the file is being read correctly (line by line), however it seems that no txns are being setup in the txn set when Common.QIF_MODE_DOWNLOAD is used. An example of the rows in the QIF file being imported are:

!Type:Bank
D2019-05-20
T-1246.19
NB8E21SQA00JA004V
C
PJoe Bloggs
L
MSaldo beheer xxxxx318
^
D2019-05-20
T-86.56
NB8E20AG2JK000023
C
PAlbert Heijn
L
MABC 20-05-2019 16:52
^

  1. System closed this discussion on 27 Aug, 2019 02:50 PM.

  2. rapscallion re-opened this discussion on 17 Sep, 2019 04:41 PM

  3. Support Staff 1 Posted by Sean Reilly on 17 Sep, 2019 09:04 PM

    Sean Reilly's Avatar

    Hi Mike,

    Ah, sorry, those QIF import functions are outdated and probably won't work very well anymore. We've reimplemented QIF import and there's now a much better way to do it. Currently the easiest interface to it is to call getUI().importFile(fileToImport:File) or getUI().importFile(fileToImport:File, newDataSet=false, defaultDestinationAccount:Account)

    That will automatically select the best importer (QIF, text delimited, OFX) as well as the best mode (downloaded, from-another-app, etc).

    Thanks,
    Sean

    --
    Sean Reilly
    Developer, The Infinite Kind
    https://infinitekind.com

  4. 2 Posted by rapscallion on 17 Sep, 2019 11:50 PM

    rapscallion's Avatar

    Hi Sean,

    Thanks for your reply. I can see the getUI().importFile(fileToImport:File) method, but using this brings up the UI import dialog and when I then request the load Moneydance hangs.

    As I have a number of QIF files to load each time and the fact that I know which account each needs to be loaded, I would really like to use the getUI().importFile(fileToImport:File, newDataSet=false, defaultDestinationAccount:Account) method, but I don't see this as an available choice in the method completion in Eclipse - please see the attachment for details.

    Any ideas?

    Regards, Mike.

  5. 3 Posted by rapscallion on 18 Sep, 2019 07:33 AM

    rapscallion's Avatar

    Hi Sean,

    Thinking about this a little more, I would rather not have the UI present any dialog to the user at all, I would prefer instead to be able to load the QIF files directly. I noticed in an older posting on a different but similar question (see https://infinitekind.tenderapp.com/discussions/moneydance-developme...) you made a comment to the effect that there is a more direct API available?

    "...There's a deeper API that will also let you import a bit more directly, but I'd recommend going with the one above."

    Can you please let me know what that deeper API call is? In my case, I know the file type (QIF), the date format and the receiving account so I don't want to bother the user with any unnecessary UI interaction.

    Thanks in advance.

    Regards, Mike.

  6. 4 Posted by rapscallion on 26 Oct, 2019 08:09 AM

    rapscallion's Avatar

    Hi Sean,

    I have resolved my immediate issue by transforming input files into a common CSV format (rather than QIF) and then using the TabularTextImport functionality which is working well for standard bank and credit card accounts.

    Now, I still think that a similar (deeper) QIF API loader should be supported, so would still like to get that information please so I can support other account types such as investment accounts. As far as I can see, the TabularTextImport cannot support those...?

    Any info you can provide on the deeper QIF loader methods would be useful. Thanks in advance.

    Regards, Mike.

  7. 5 Posted by rapscallion on 20 Nov, 2019 05:22 PM

    rapscallion's Avatar

    Hi Sean,

    Any update on the documentation for the "deeper" api to load QIF files directly? In the meantime I am leveraging the CSV loader directly, but this doesn't give me split txn support nor does it allow import of securities data... I really need direct access to the lower level QIF loader (like it used to be supported in earlier versions).

    Regards, Mike.

  8. 6 Posted by rapscallion on 26 Jan, 2020 07:35 PM

    rapscallion's Avatar

    Hi Sean,

    Happy New Year, any chance I will get an answer on this soon?

    Regards, Mike.

  9. 7 Posted by rapscallion on 04 Mar, 2020 11:36 AM

    rapscallion's Avatar

    I suspect that you guys are not interested in this topic, so I guess we can close it.

    It's a pity that I will need to do low level transaction insertion one row at a time through the API rather than leveraging the inbuilt QIF loader directly via the API. This will be required because as far as I can tell the CSV loader is not capable of loading stock data and/or split transactions.

  10. Support Staff 8 Posted by Sean Reilly on 04 Mar, 2020 11:44 AM

    Sean Reilly's Avatar

    Hi Mike,
    Sorry, I've been spread a bit thin lately. We definitely need to expose this. I ran into a minor obstacle with providing access to the underlying importing function, but I really should add a convenience method in the meantime to import from various formats with a UI/no-UI flag.

    Please do keep this open!

    Thanks,
    Sean

    --
    Sean Reilly
    Developer, The Infinite Kind
    https://infinitekind.com

  11. 9 Posted by rapscallion on 06 Mar, 2020 05:13 PM

    rapscallion's Avatar

    Hi Sean,

    Thanks for this consideration, that would indeed help ā€” would you please be able to let me know when this might be ready to test? Iā€™d be happy to help.

    Regards,

    Mike.

    Michael P. Leznar
    Director | Leznar Software & Services BV
    www.leznarsoftware.com
    Tel: +31 6 2185 6272

  12. 10 Posted by Dan P on 28 May, 2020 04:48 PM

    Dan P's Avatar

    Sean and All,
    I'm glad I found this topic and wish to add my support also. I've developed a Java program to access my bank and download QFX files by date range. I run it each day and manually import the files to MD. It really does save me time.
    I was starting to develop an extension to import the stream directly, but as you have indicated, there is no access to the ImportFile API without the UI.
    I will continue to develop my extension in the hope that when I have it ready, the non-UI API will be available.

    Keep up the good work!

  13. 11 Posted by Dan P on 30 May, 2020 06:50 PM

    Dan P's Avatar

    Sean,

    I may be a little daft but I cannot find any API documentation to implement what you suggested to Mike.
    "Currently the easiest interface to it is to call getUI().importFile(fileToImport:File) or getUI().importFile(fileToImport:File, newDataSet=false, defaultDestinationAccount:Account)"

    Can you give me a push in the right direction?

    *** I figured it out and it actually works.

  14. System closed this discussion on 29 Aug, 2020 06:50 PM.

  15. rapscallion re-opened this discussion on 14 Sep, 2020 09:12 AM

  16. 12 Posted by rapscallion on 14 Sep, 2020 09:15 AM

    rapscallion's Avatar

    s there an update to the 'without UI' request? Reference: Sean's comment above

    Sorry, I've been spread a bit thin lately. We definitely need to expose this. I ran into a minor obstacle with providing access to the underlying importing function, but I really should add a convenience method in the meantime to import from various formats with a UI/no-UI flag.

    Please do keep this open!

  17. Support Staff 13 Posted by Sean Reilly on 18 Sep, 2020 11:51 AM

    Sean Reilly's Avatar

    Hi there,

    I'm sorry, there isn't yet. I do hope to add this in the future, but for right now the per-import UI is necessary to customise the import settings.

    Thanks,
    Sean

    --
    Sean Reilly
    Developer, The Infinite Kind
    https://infinitekind.com

  18. 14 Posted by kim.aldis on 26 Nov, 2020 01:11 PM

    kim.aldis's Avatar

    Hi, I'd like to add my vote to a GUI-free qif import. I currently download and import transactions from a number of accounts for which, given how tedious doing this by hand is, I've written a Python import script. It works well but I'd much rather use provided import methods if possible.

  19. 15 Posted by tgilbert666 on 18 Dec, 2020 08:47 AM

    tgilbert666's Avatar

    +1 from me

  20. 16 Posted by Stuart Beesley ... on 02 Feb, 2021 11:29 PM

    Stuart Beesley (Mr Toolbox)'s Avatar

    Hi all, I have created a demo python script which simulates calling the importFile() function as referenced above... (It make the same calls, but via this script... I can see why it's a bit tricky to expose...) However, you can choose in the script whether to run headless with no UI/popups or to have the popups. If you choose popups then it asks what file and also allows you to choose the import type flag; and then runs as normal... The script also allows you to pre-set the Downloaded or Migrated flag... I haven't tested the import that deeply, but basic imports seem to work OK.. Take a look, I am sure you will get the gist. You can set other (spec) settings, but you will have to play with this... NOTE: the standard function runs each step on it's own thread... I haven't done this; so you will have to do this if you want/need it.. Use at your own risk. Happy to take questions etc? Use at your own risk.... (I think the biggest risk is that the methods could change in the future). SCB

  21. Support Staff 17 Posted by Sean Reilly on 03 Feb, 2021 09:43 AM

    Sean Reilly's Avatar

    Hi Stuart,

    This script is a great! You might want to remove the part that calls the import settings UI since if you're using the UI you won't need the script. Not to mention that the UI parts are more likely to change.

    One other change I'd suggest would be to setup the "import spec" object at the beginning with all of the main parameters (target account, hasTargetAccount, date field order, sourceType, encoding, decimal point). When doing the prescan the importer can override some of the parameters (sourceType, date field order, decimal point) if it detects that the given parameters don't make sense, so you might also want an option to force them to your settings after the prescan. For example, the sourceType (which indicates whether a QIF file was downloaded or not) is usually adjusted by the prescan if it detects signs that indicate whether the QIF was downloaded or not, but you may know better.

    Also, IMO it's probably a good idea to not save preferences associated with the import. That done from the GUI in order to make repetitive actions easier, but in this case the script takes care of that.

    Thanks,
    Sean

  22. 18 Posted by Stuart Beesley ... on 03 Feb, 2021 09:54 AM

    Stuart Beesley (Mr Toolbox)'s Avatar

    Thanks Sean... It was a bit of guesswork.. I'll incorporate your great suggestions. Thanks!!

  23. 19 Posted by Stuart Beesley ... on 03 Feb, 2021 01:06 PM

    Stuart Beesley (Mr Toolbox)'s Avatar

    I've updated the script to include more commentary on the methods of setting the spec; and also save/load preferences (as Sean suggested).. I have left these as comments so that the programmer can see where/how to do this.. I've left the UI methods in, not to be used, but to demonstrate how MD normally calls this function for a comparison. Really this is an educational script for a basic start point.... Let me know how you get on with it..?

  24. 20 Posted by Stuart Beesley ... on 03 Feb, 2021 10:26 PM

    Stuart Beesley (Mr Toolbox)'s Avatar

    Sean, a quick question. Is there a way to tell the importer to just create the structures (accounts, cats, securities, currencies etc) without actually adding the data (as the old qif importer could do years ago? Thx.

  25. Support Staff 21 Posted by Sean Reilly on 03 Feb, 2021 11:45 PM

    Sean Reilly's Avatar

    No, sorry the ability to import only accounts and categories but not transactions was removed when we switched to the new system. I wanted to simplify the import process for the 99% of cases that didn't need it. It is fairly easy to edit a QIF file to remove everything but the accounts and category lists.

    Thanks,
    Sean

  26. 22 Posted by Stuart Beesley ... on 05 Feb, 2021 04:58 PM

    Stuart Beesley (Mr Toolbox)'s Avatar

    @rapscallion, @DanP, @tgilbert666, does this example script help you? I wrote it for you?

  27. 23 Posted by Dan P on 05 Feb, 2021 05:15 PM

    Dan P's Avatar

    Stuart,

    My apologies. When I first read this thread, I got confused between QIF and QFX. My extension uses the QFX import function which works perfectly. No UI.
    I do 'screen scraping' on two of my accounts as they offer QFX download but no direct connection.

    You do great work! I love the Toolbox.
    I did a test for a friend using QIF last year. I will try your script just to test it out.

    Dan

  28. 24 Posted by Stuart Beesley ... on 05 Feb, 2021 05:19 PM

    Stuart Beesley (Mr Toolbox)'s Avatar

    Great! I think the code can work out which import you need and then call the relevant importer.

  29. 25 Posted by Stuart Beesley ... on 05 Feb, 2021 06:03 PM

    Stuart Beesley (Mr Toolbox)'s Avatar

    a better and final demo/educational version

  30. 26 Posted by Dan P on 05 Feb, 2021 08:03 PM

    Dan P's Avatar

    Stuart.
    Your script seems to work beautifully. I imported QIF, CSV, and QFX and all loaded perfectly.
    I did have some issues understanding the CSV column names, but that ended up being a data problem. (Being lazy, I just grabbed some older files.)

    Thanks!

    Dan

  31. 27 Posted by Stuart Beesley ... on 05 Feb, 2021 08:12 PM

    Stuart Beesley (Mr Toolbox)'s Avatar

    👍

  32. 28 Posted by rapscallion on 07 Feb, 2021 04:20 PM

    rapscallion's Avatar

    Unfortunately, this does not help me because I am looking for this functionality in Java so I can include it in my own loader module (not in Python). Unless I am missing something, there does not seem to be a similar 'getFileImporter' method for Java, only a tabular file importer which I am already using. However, the problem is that this doesn't have all the capilities to load everything that can be done via QIF, e.g. it will only do regular back account type loads, not stock investment accounts.

    I have to say that I am rather disappointed in IK for their reluctance to get this interface (that BTW used to exist in the software years ago...) exposed properly once again in Java. I am now faced with a decision on a paid upgrade to Moneydance 2021, and wondering whether my request for this interface in Java will ever be honoured. Sean, is there ever going to be some time for you guys to get a similar interface exposed for Java?

    Regards, Mike.

  33. 29 Posted by Stuart Beesley ... on 07 Feb, 2021 04:34 PM

    Stuart Beesley (Mr Toolbox)'s Avatar

    Hi Mike. Python in MD is Jython. This is Java based. I am calling the MD internal Java routines directory. You should be able to port my code back into Java (as I ported it out of the original MD Java). I see no reason why it wouldn’t work for you in your own Java. As for investments, if the MD importer does those then this will work. If it doesn’t then I can’t help you. S

  34. 30 Posted by rapscallion on 07 Feb, 2021 11:28 PM

    rapscallion's Avatar

    Hi Stuart, I think I might know why I had a problem... I was using an older Moneydance.jar file form 2019 that did not have the FileImporter class. Duh!!

    I updated the jar to the latest 2020 version and found the class... my apologies for suggesting your solution wouldn't help, I am sure it will now. Many thanks.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ā†© Submit the comment

You can use Command āŒ˜ instead of Control ^ on Mac