WebConnect downloader

moneydanceuser's Avatar

moneydanceuser

13 Jun, 2011 06:07 PM

Here's an update to the WebConnect automated downloader idea originally discussed on the moneydance-info mailing list...

I've updated the code at http://code.google.com/p/web-banking-assistant/ to include support for Chase Bank. In many ways writing support for Chase Bank was easier than it was to write support for Pentagon Federal Credit Union. With this update I am hoping to spur the developer community to do two things: 1) Get a better idea of what it takes to add support for a new bank, 2) Compare/contrast the code for the two banks to see if there is a way to make it more generic so additional banks can be more easily supported or whether there is a better way to accomplish the goal of automating downloads of QFX files.

My most immediate need is someone to help me write makefile rules to build the project. (I'm using GNU Make 3.81) If you can help please ping me offline.

  1. 1 Posted by moneydanceuser on 13 Jun, 2011 06:25 PM

    moneydanceuser's Avatar

    To answer some questions that were asked on the mailing list:

    Q: Is this project open source?
    A: yes, and it is free for anyone who wants to use it.

    Q: What programming language is used?
    A: Currently the core functionality is written in JavaScript. A JavaScript engine is needed to run the code. I am currently using Fake.app as the engine. http://fakeapp.com/. I'm willing to change to whatever language and engine makes it easier to achieve the goal of automated downloads.

    Q: Which operating systems are supported
    A: The core JavaScript should run on an OS and any JavaScript engine.

    Q: Is this project going to be specific to downloading only QFX files?
    A: This project simply downloads a file from a web site after following several explicit steps to navigate to the correct page. This project should be totally agnostic to the type of data being downloaded.

    Q: Is there a plan to make this project into a Moneydance plugin?
    A: That sure would be the most useful application of this project, but its too early to commit to creating a plugin.

  2. 2 Posted by Angie Rauscher on 15 Jun, 2011 02:42 PM

    Angie Rauscher's Avatar

    Wow, this is just fantastic. We're currently exploring what it would take to support Web Connect, and this is great information to have. I've sent a copy of this thread to our direct connection specialist (who is also working on the web connect project).

    Thanks again for sharing your work with other users,

    Angie Rauscher
    Moneydance Support

  3. 3 Posted by rob on 15 Jun, 2011 05:12 PM

    rob's Avatar

    The only catch is that it relies on 3rd party commercial software (Fake, ~$30). Regardless, I'm eager to try this, Steve. Nice work!

  4. 4 Posted by trevor on 15 Jun, 2011 07:50 PM

    trevor's Avatar

    Is this available for anyone to play with/try out?

  5. 5 Posted by rob on 20 Jun, 2011 07:22 PM

    rob's Avatar

    Yes, azguy:
    Step 1, download Fake.app (listed above, for Mac only)
    Step 2, download the Javascript source (also listed above)

    This is not a polished plugin yet. It's still early stages.

  6. 6 Posted by moneydanceuser on 27 Jun, 2011 08:41 AM

    moneydanceuser's Avatar

    Here's some good news for people who are not using a Mac and/or do not want to buy Fake.app. The JavaScripts I have written will work in Firefox via GreaseMonkey. This gives a totally free solution for everyone on most any platform supported by Moneydance.

  7. 7 Posted by moneydanceuser on 07 Jul, 2011 05:58 PM

    moneydanceuser's Avatar

    I've identified potentially two additional JavaScript engines that could run these transaction download scripts:

    1. Node.js
    2. HTMLUnit

    Node.js - http://nodejs.org/ - is a GUI-less JavaScript runtime environment. It has the ability to connect to web sites and inject JavaScript on that site. Conceivably node.js could be used as the engine for our web connect scripts. There are at least four different web scraping plugin frameworks for node.js that could potentially make it easier to support many different banks with less custom code per bank.

    HTMLUnit - http://htmlunit.sourceforge.net - is basically a GUI-less browser (with support for JavaScript) completely implemented in Java. Conceivably it could also be used as the engine for our web connect scripts. The added advantage to HTMLUnit is that it is written in Java which makes it much more attractive for creating a Moneydance plugin.

    Of all the JavaScript engine options so far, Fake.app, GreaseMonkey with FireFox, node.js and HTMLUnit, the most promising option from a Moneydance perspective is HTMLUnit. Its slight disadvantage is that it would require writing a lot more foundation code than it would to use any of the other solutions. Still, I think HTMLUnit is the way to go.

    What do you think?

    Any volunteers to write the Java foundation for running our transaction download scripts?

  8. 8 Posted by rob on 06 Sep, 2011 06:07 PM

    rob's Avatar

    Have you guys been successful with Fake? I’m having a hard time identifying certain bank website’s underlying variable names to insert javascript. Just wondering if anybody is having better progress than me.

  9. 9 Posted by rob on 17 Feb, 2012 05:20 PM

    rob's Avatar

    Anyone still poking around at this stuff?

  10. 10 Posted by Angie Rauscher on 17 Feb, 2012 05:30 PM

    Angie Rauscher's Avatar

    Hey Rob,

    Our connection specialist, Jessica, has been working on this project for a while. I'll ask her to chime in on this thread if she has any additional info- last I heard she was making good progress but our tool wasn't ready for external testing.

    Angie Rauscher
    Moneydance Support

  11. 11 Posted by Jessica Little on 17 Feb, 2012 07:53 PM

    Jessica Little's Avatar

    Hi Rob,

    I'm not working on this extension, but am definitely still working on web connect in Moneydance proper. We're aiming for a testable version soon. We had a slight setback recently that has been difficult to work around, but I hope to be back on track soon. I'll definitely let everyone know when a beta is available.

    Jessica Little
    Moneydance Support

  12. 12 Posted by Ramsay Zaki on 15 Jul, 2014 09:03 AM

    Ramsay Zaki's Avatar

    Is anyone still working on this (or the ability to do Web Connect in MoneyDance itself)?

  13. Support Staff 13 Posted by Sean Reilly on 13 Aug, 2014 10:02 PM

    Sean Reilly's Avatar

    Hi Ramsay,

    Sorry for the very delayed reply. We have recently renewed an effort to work on making it easier to extract data from web pages and get it into Moneydance. We've got a basic browser extension working that detects and extracts financial data from many web sites and can (almost) send it directly over to Moneydance for a seamless import.

    We at TIK really need to set up a new developers forum. I've just created one here which we can use going forward for public discussions:
      https://infinitekind.zendesk.com/hc/communities/public/topics/200102485-Development

    For private messages please feel free to email us at [email blocked] or me personally at [email blocked].

    Thanks!
    Sean

  14. Sean Reilly closed this discussion on 13 Aug, 2014 10:02 PM.

  15. Sean Reilly re-opened this discussion on 13 Aug, 2014 10:02 PM

  16. 14 Posted by Gary on 20 Aug, 2014 03:03 AM

    Gary's Avatar

    I'm not a Moneydance user 'yet' but looking into it. I'm having trouble understanding what this thread is about, but can someone confirm the the program can download transactions from BofA, Citi Cards and Capital One Bank and credit cards at this point? It seems that iBank can't do Capital One for me. Thanks!

  17. System closed this discussion on 31 Mar, 2015 03:38 PM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac

Recent Discussions

29 May, 2024 07:01 AM
29 May, 2024 06:26 AM
29 May, 2024 04:47 AM
29 May, 2024 02:38 AM
29 May, 2024 12:43 AM