Quote Loader Price Errors from Yahoo Today
Today, after working perfectly every day, the Quote Loader extension reported errors on all prices obtained on both my Windows 11 desktop computer and my MacBook Air M4, both with latest OS installed. I note that nobody else has reported that here or on a Google search.
The source for all prices is Yahoo. The same result was obtained after changing to Yahoo HD. Changing to FT yielded no errors, although yesterday's prices were recorded, probably due to time zone difference. It seems the problem lies with Yahoo and not Quote Loader, but I'm puzzled that no-one else has reported it.
Showing page 4 out of 4. View the first page
Comments are currently closed for this discussion. You can start a new one.
Keyboard shortcuts
Generic
| ? | Show this help |
|---|---|
| ESC | Blurs the current field |
Comment Form
| r | Focus the comment reply box |
|---|---|
| ^ + ↩ | Submit the comment |
You can use Command ⌘ instead of Control ^ on Mac
91 Posted by Timppa on 21 May, 2025 12:35 AM
TIK won't know if it's viable unless they try. They certainly have more "market power" than you or I do as individuals and AV's entire product seems to be aimed at software developers in the financial space. You and I are not their target audience. TIK is.
And not to be selfish, but AV meets my needs and those of a large portion of the user base. I don't think we should be allowing perfection to be the enemy of better. I'm sorry there isn't a turnkey solution for your country's financial exchanges. There is for mine. This is a feature that could and should be added to the + product, which is already North American centric, and nobody is forcing you to subscribe.
The CSV path is allowed and viable, if someone wants to engage in that as a tedious manual undertaking, but doing it via automated means is 100% a violation of that TOS. I repeat my statement that we have no business evading Yahoo!'s blocks. It's disrespectful at best and illegal at worst.
92 Posted by david.d on 21 May, 2025 12:47 AM
To take a straight Philadelphia lawyer-ish view (no, I'm not one, but I've known a lot), it's not "automated" if you don't run it on a schedule. If I click that "Get Prices" button, that's a manual (if scripted) process, not automated.
It's easy to play language games, if you want to be really literal, and that's what lawyers are paid to do. Don't let them intimidate you; you're not using their data in any illegitimate way or in any way to Yahoo!'s detriment. Indeed, if I couldn't use it for this purpose, I might not use it at all, and Yahoo! would miss the ad revenue that I bring.
Yahoo! makes that data available for the private and personal use of its users, in the plain meaning of its service offering, and this qualifies as such. And the fancy legalese in their user agreements is often unenforceable in many states.
93 Posted by Timppa on 21 May, 2025 12:55 AM
That argument goes out the window when there's an entire public discussion where people are discussing -- and implementing -- countermeasures to Yahoo!'s blocks.
You can get creative in your reading of their language if you'd like. Their actions demonstrate that they do not allow or desire scraping at this scale.
If it was just you or I doing it privately, via wget and scripts created for our private use, nobody would notice or care. This isn't that though. This is an extension provided to countless people, signed by TIK, and distributed via their systems. The Yahoo! code within the extension is 100% a violation of the Yahoo! TOS. The scale at which it happens is obviously enough to attract Yahoo!'s attention. This is, what, the third or fourth time they've tried to block it?
This is not okay. If people want to figure it out on their own, with scripts they create and don't distribute, Godspeed. TIK should not be condoning this, implicitly or explicitly.
94 Posted by Mike Bray (Quot... on 21 May, 2025 08:53 AM
I think it is correct that use of Quote Loader in its current form does 'stretch' the Yahoo TOS agreements, however, the Internet is full of such cases. If Yahoo just used a tokenised call api such as AV and others, they can control the amount of traffic hitting their servers and even make money out of the few people who have high volumes of quotes.
For example there is a new company MarketData.app who provide a tokenised API where the first 100 quotes a day are free. More quotes need to be paid for but the prices are quite small. They are US only.
If Yahoo choose to not do this then chasing their changes is not viable and one of the reasons I have stepped back from supporting QL.
A little bit of history might be helpful. The first extension I wrote was Security Price Loader which takes a CSV generated from my Investment accounts and loads the prices. This was followed by Security History Loader which loads multiple prices per ticker. At the same time a person called Lee Hung had an app that down loaded prices from Yahoo and loaded them into Microsoft Money. I was put in touch with him and we collaborated to make a pair of extensions. Lee's extension downloaded the prices and my extension loaded them into MD. When Sean became more forceful about reviewing code before verifying an extension Lee dropped out and I took on the extra code. This was the birth of Quote Loader that we know today. I think the future of Quote Loader is to use tokenised APIs rather than screen scraping. Sean has told me he will move my extensions to the Open repository and thus they will be available for all to use. Before that you can find the latest versions at https://github.com/mrbray99/moneydanceproduction/tree/main/downloads. There is a wiki at https://github.com/mrbray99/moneydanceproduction/wiki but I expect this will become out of date.
95 Posted by sth on 21 May, 2025 10:57 AM
Mike B.,
And thank you for the work on QL and integrating it into a fine tool. I agree that chasing Yahoo's continual changes of trying to allow manual downloading but not "automated" downloading will be a never ending game of whack-a-mole. Yahoo is walking a fine line here.
MarketData.app is interesting. The 24 hour delay on prices may be a killer for the free option and the 12/mo is fairly steep for those who are just not active traders. Not clear if they cover foreign exchanges.
Mike, thanks again!
-Scott
96 Posted by kmenningen on 21 May, 2025 12:55 PM
I agree that scraping Yahoo prices is not a good idea, which sounds like a consensus in this discussion. It's one reason why I stopped working on Quotes and Exchange Rates Updater.
Is there a way to import a CSV into MD to update security prices? Excel has a relatively recent feature to mark cells as "Stocks" and then get various fields for the stock (I counted 31 different fields just now) including high/low/close/volume. I think it uses Bing to get the prices. In Excel, all you do is go to Data then "Refresh All" and everything updates. It's free and totally sanctioned.
So the process would be to create an Excel spreadsheet with your tickers of interest that has date/open/high/low/close/volume. Update it with "Refresh All". Then save as a .csv. Then import the .csv into Moneydance (my question above is how this could be done).
Albeit a little clumsy, it's a free option and is completely supported by Microsoft.
97 Posted by Rich Braun on 21 May, 2025 02:31 PM
We're all investors here. We represent Clout with a capital-C. I agree with Timppa above: The Infinite Kind won't know until it actually TRIES to use that clout to negotiate deals.
But you don't do this by begging data brokers like Yahoo, Bloomberg or Financial Times.
Go right to where the money is. And where's our money? Ameritrade, Morgan Stanley, Charles Schwab, Fidelity, Interactive Brokers, HSBC et al. Those companies know we will trade more if we have better visibility across our assets, and the cost to them of providing this data is peanuts.
With my Schwab credential, I ought to be able to pull down prices of everything I might ever want to to buy at Schwab. It's in their interest to do so.
There IS a way to solve this problem, without tacking on a $20USD monthly fee. If it's NOT solved, then sales of Infinite Kind's products will go down.
Am I making sense?
98 Posted by robnich1 on 28 May, 2025 12:22 AM
For some reason, this discussion was made Private, but not by me, so I made it Public again, as I have some new and, to me, baffling information. Yahoo has been working perfectly for me since I updated QL to v.321706, including today, on my Windows 11 computer. However, when I opened Moneydance on my MacBook Air, the automatic run of QL reported 18 (out of 18) errors, i.e. no updating at all. This is the very same Moneydance data file that was on my Windows computer, since I had transferred it to the MacBook after the price update Friday, the last market day. After I transferred today's newly updated data file, all prices were correct. In other words, security prices automatically updated correctly today in the Windows computer but not on the MacBook, although it was the identical data file on both.
As there may be no response or reaction here, I may repost this in the Investment section of the discussions.
99 Posted by dwg on 28 May, 2025 12:28 AM
@kmenningen Mike also created the Security Price Loader extension which imports a CSV containing prices for a specific date, he also has created the Security History Loader that can import prices for multiple dates
Moneydance itself can import CSVs of prices, but it requires a separate CSV for each security.
100 Posted by dwg on 28 May, 2025 12:32 AM
@robnich1 look at the version of QL on the Mac. Copying the data file does not copy the extension.
101 Posted by dwg on 28 May, 2025 12:46 AM
@robnich1.
If you did not make the extension private then only the support folks could have done so, they would have to explain their actions.
For one, I am not seeing anything that could be deemed confidential in this discussion.
102 Posted by Stuart Beesley ... on 28 May, 2025 06:29 AM
@robnich1, @all, please, if you get unexplained errors, then immediately grab help/show console, and upload here. Otherwise, it’s fumbling in the dark.
Transfer of dataset sounds suspicious. Why / how are you doing this, and how are you keeping the two computers in sync? Personally, I would check the extension versions installed.
103 Posted by Stuart Beesley ... on 28 May, 2025 08:36 AM
PS. Preview 322007 is a better build.
104 Posted by robnich1 on 28 May, 2025 03:07 PM
@dwg, You are correct - the QL version is not the same, it is 321605 on the MacBook and 321706 on the Windows 11 computer. That would seem to explain the issue I observed.
I will stop transferring the Moneydance data file from one computer to the other and will update each separately, as I should have been doing from the outset. I will also install Preview 322007 as suggested by Stuart Beesley
Sorry to add confusion to an already difficult problem. I will fall back on my age (3 months short of 90 years old) as an excuse for my deficiencies :-)
105 Posted by sth on 28 May, 2025 03:29 PM
@robnich1, it is probably better to sync the two systems between the two computers using the built-in sync through Dropbox or iCloud. In either usage the file is well encrypted but synchronizes transactions etc. It is a lot easier than copying the file back and forth, or trying to keep two files synchronized and up to date manually.
https://infinitekind.tenderapp.com/kb/syncing-and-sharing-data
(NOT IK Support)
106 Posted by robnich1 on 29 May, 2025 12:56 AM
@sth, Thank you for the suggestion. Any reason not to use the Shared Folder option in the built-in sync function of MD? I do have such a folder with which I transfer files from my desktop computer to the MacBook I am not really clear, though, how syncing the Moneydance data file differs from simply transferring it from one computer to another either by Wi-Fi or USB key.
107 Posted by dwg on 29 May, 2025 02:53 AM
Shared folder works, it is more for LAN solutions and it does not support mobile devices so most use Dropbox or iCloud because of that.
Copying the data file also works, but there is more risk of human accident. I mostly use my desktop system for my own Moneydance data so do not really have a need for syncing but if I do go away I indeed copy it to a notebook and copy it back on my return.
108 Posted by derekkent23 on 29 May, 2025 06:59 AM
I am not support staff, just a user.
Hi robnich1
Using Moneydance syncing means no more having to copy data file and the possibility of human error. It a one time setup. Once set up add a transaction on one computer and if the other is running Moneydance it will as if magic appear within one minute. If the other computer is not running Moneydance then when you open it the work you were doing on the other computer will appear.
Hope this helps.
109 Posted by robnich1 on 30 May, 2025 12:53 AM
Thank you Derek and dwg. Very helpful.
System closed this discussion on 29 Aug, 2025 01:00 AM.