Home → Problems →

Update: Quote Loader, Getting Security Prices

• Edit

Mike Bray (Quote Loader Author)

16 Apr, 2024 07:17 AM

As you may know the format of Yahoo web pages has changed. This has broken Quote Loader and Quote and Exchange Rates Updater for the Yahoo source. This is not a simple fix.

Quote Loader has been recommended as an alternative using the FT source. Often the ticker used on FT is different than that used on Yahoo so just changing the source may not work. To help you overcome this these steps might help.

1. Set up Quote Loader with a source of FT against each security
   
2. Click on Save Changes
   
3. Click on Get Prices
   
4. Once all prices have been returned (green) or shown in error (red). Check the successful prices to see if the new price makes sense. What can happen is that the ticker used on Yahoo could be used for a different security on FT, thus the price will be quite a bit different. Identify those securities which are wrong and add them to the list of failed securities.
   
5. Use your browser to got to ft.com (https://markets.ft.com/data)
   
6. For each security use the Search securities box (top right) to find the page for your security (eq. Apple Inc). Once you have found it ft.com will display the name of your security with the ticker used by ft.com (eg. aapl:nsq).
   
7. Copy this ticker and paste it into Alt Ticker box against your security, make sure the Exch Mod box is empty by double clicking on the box and choosing the first entry 'Do not use an exchange'
   
8. Once you have updated all failing securities click on Save Changes
   
9. Click on Get Prices

That should solve your issues, if you still have issues make sure there are not any spaces or wrong characters in the Alt Ticker box.

1. 1 Posted by DrgnMastr on 16 Apr, 2024 01:12 PM

   

   Thank you for the detailed reply. Unfortunately, this process doesn't work for FT HD.

   I certainly have appreciated this extension over the last couple of years, and do hope that you find the solution to the Yahoo and Yahoo HD issue. I will hold off making the changes to FT, as I really do need the history as well, and that is not working.

   Further compounding the issue, is that I made use of the Exch Mod field extensively with Yahoo, as my securities trade on the Canadian TSX exchange.

   Thank you for all the work on this extension. It certainly is and has been the one extension that I relied on the most.

   • Edit

2. 2 Posted by derekkent23 on 16 Apr, 2024 02:53 PM

   

   I am not support staff, just a user.
   Hi DrgnMastr

   FT HD does not support historical data for Stocks, only Funds, nor does it support Historical data for Exchange Rates.

   So for Stocks and Exchange Rates use FT and run every day. I run the next morning before the market opens.

   Any Securities that return an error with FT as the source then post your base currency, the Ticker, full Security Name and last price you have and its traded currency or just post a screen shot of the Quote Loader windows after you click GET PRICES.

   Hope this helps.

   • Edit

3. 3 Posted by DrgnMastr on 16 Apr, 2024 02:57 PM

   

   Derek,

   As I indicated in another thread, I have several securities that trade on the TSX only. Your solution is a viable one but onerous for me, as I only open MD when I need to make changes, not every morning. Hopefully, a fix will be found soon, otherwise I will have an incredible amount of work getting my securities working again.

   Cheers!

   • Edit

4. 4 Posted by davidwoh on 17 Apr, 2024 03:46 AM

   

   After several failed attempts, I was able to convert over to FT. To do so, change your security symbols to match FT's requirements. Then go to Quote loader and select FT for download. FT HD does not work for most symbols. Setting the download in Quote Loader before changing symbols also does not work. In other words, I should have followed Mike's instructions instead of assuming that I knew what I was doing.

   • Edit

5. 5 Posted by dwg on 17 Apr, 2024 03:54 AM

   

   I believe that FT HD does not support shares only funds.

   • Edit

6. 6 Posted by okichatan on 17 Apr, 2024 03:55 AM

   

   Thank you Mike. Your fix worked for me. 👍

   • Edit

7. 7 Posted by david_nedved on 17 Apr, 2024 12:21 PM

   

   Has anyone been able to find a ticker symbol that works with FD for gold and silver bullion by the ounce? On Yahoo I had been using "GC=F" for gold and "SI=F" for silver which had been working until their recent change that locked us out. FT does not accept these nor can I find anything using the search engine on ft.com as referenced above.

   • Edit

8. 8 Posted by DrgnMastr on 17 Apr, 2024 12:26 PM

   

   David,

   I wish I had known about those symbols. I have been inputting that price, once a week, now for years from a web site that has current bullion prices in any currency.

   • Edit

9. 9 Posted by don.armstrong on 17 Apr, 2024 02:58 PM

   

   I hold a number of Canadian funds and ETFs, and it appears that quotes for these are not available from FT, but only from Yahoo. Examples iShares MSCI Canada Min Vol ETF (XMV on Toronto exchange), or CI High Income Fund A CIG786 (Yahoo 0P000070CX.TO). If you know how to access these from FT, please let me know, otherwise I am dependent on Yahoo

   • Edit

10. 10 Posted by DrgnMastr on 17 Apr, 2024 03:15 PM

    

    Don,

    I feel for you. I have several Canadian funds and ETFs that I could never download their prices even from Yahoo. I haven't found one of them on Yahoo or FT, so took a different course. What I ended up doing with those was to create a Morningstar portfolio with them, and Saturday mornings, I manually input their prices using the "Security Price Entry" extension. I would love to have a way of taking the data from Morningstar directly into MD, but have not found a way yet.

    • Edit

11. 11 Posted by matt8375 on 17 Apr, 2024 10:22 PM

    

    Great, thanks for your help.
    I'm having trouble with Canadian ETFS, specifically COW & ZAG

    Any ideas?
    Theya re listed on TSX

    • Edit

12. 12 Posted by derekkent23 on 17 Apr, 2024 10:44 PM

    

    I am not support staff, just a user.
    Hi matt8375

    It looks like one of your two securities is supported by FT. See screen shot and note with FT selected as the source I have typed in a Alt Ticker COW:TOR:CAD

    Hope this helps.

    • matt8375.png 53 KB
    • Edit

13. 13 Posted by carrbigdog on 18 Apr, 2024 05:58 AM

    

    I appreciate all the efforts to provide a work around for this issue, but since I am an investor interested in longer term changes, going through the efforts to identify different sources/symbols is not something I am looking forward to, that is, if there might be a change to the interface that would re-ignite the current configuration in the next month or so. Has Yahoo published the interface requirements that changed? Is there a way to donate a few bucks to facilitate any updates? Am I the only one that would contribute to keeping this extension active and up to date?

    • Edit

14. 14 Posted by derekkent23 on 18 Apr, 2024 07:03 AM

    

    I am not support staff, just a user.
    Hi carrbigdog

    Just globally change the source to FT and see how many prices are returned.
    Its simple to change back to Yahoo.

    The thing to remember is if FT requires a different Ticker to that used by Yahoo, using the Alt Ticker feature in Quote Loader the original Ticker is still there and used if you switch back to Yahoo. So nothing lost.

    If the need to change the Exch Mod field comes into play, then you may need to make changes when returning to Yahoo. So if you do have to change this to get FT to work, make a note of the original setting. It up to you if you take advantage of these features.

    Mike is working on a fix, but if possible, it may take some weeks. Quate Loader was developed in his spare time many ,many years ago and he continue to provide ongoing support. He provides this service to other Moneydance users for free.

    Yahoo provides no information and never has. Mike has to experiment and test in the hope of finding a fix.

    Have Fun, Stay Safe.

    • Edit

15. 15 Posted by KY on 19 Apr, 2024 01:13 AM

    

    Thank you for this very valuable support.

    I managed to get Quote Loader working based on Mike Bray's instruction and all my securities are returning green or red.

    However, what i am not getting is the prices of the securities updated on the Stock Prices on Moneydance and hence the value of my portfolio is unchanged

    How do i solve this issue?

    • Edit

16. 16 Posted by dtd on 19 Apr, 2024 02:39 AM

    

    Do you click on "Get Prices", "Select All Lines" and "Save Selected Lines" in that order?

    What is still red - Derek could probably help in figuring out how make those green.

    • Edit

17. 17 Posted by KY on 19 Apr, 2024 06:19 AM

    

    Thank you dtd! Its working now.

    • Edit

18. 18 Posted by derekkent23 on 19 Apr, 2024 07:38 AM

    

    I am not support staff, just a user.
    Hi KY

    Looks like you are on the road to success.

    However you state “my securities are returning green or red.”
    If any NEW PRICE is red there is still work to be done.
    If this is the case post a screen shot of the of the Quote Loader window after you click GET PRICES.

    Have Fun, Stay Safe

    • Edit

19. 19 Posted by mjansic on 19 Apr, 2024 10:03 PM

    

    I, like many others, have been using Yahoo to download my security prices with Quote Loader, and been VERY happy with it. Due to the current issue, I switched to FT, as suggested, and removed the exchange. The only issue I have is that I cannot get the prices until the next day. Yahoo was always less than two hour after closing. Am I doing something wrong or is that just the FT way? Thanks for any answers.
    Marc

    • Edit

20. 20 Posted by dwg on 19 Apr, 2024 10:59 PM

    

    Yahoo is based in the U.S. FT is based in the U.K. I would expect that FT is probably doing updates at times to more suit Europe.

    • Edit

21. 21 Posted by davidwoh on 20 Apr, 2024 02:30 AM

    

    I think that it is a matter of timing when you do the download. London is 5 hours ahead of EST and the markets close at 1130 in the East Coast so I suspect that the issue is when their system grabs the data and not something driven by UK requirements. I tend to pull later (tonight at 10pm but I know I have done it as early as 7). I never have a problem with getting yesterday's data.

    • Edit

22. 22 Posted by dwg on 20 Apr, 2024 02:43 AM

    

    I think we can say that these organisations will update their data based on their systems schedule, it is all going to be automated, so it will come down to their scheduling

    • Edit

23. 23 Posted by anonymous on 20 Apr, 2024 04:52 AM

    

    FYI, if anyone's interested in fixing the yahoo quote fetcher, the key breakages are in the price value attribute (now uses 'data-value') and the currency span (which no longer exists as it did before) -- it can however be fetched from a different tag in the page:

    In the file GetYahooQuote.java that's available from the github repo, this is what i have for the parseDoc function:
    

     private void parseDoc(Document doc, QuotePrice quotePrice) throws IOException {
            String query = null;
            Element docElement = null;
            try {
                String formattedTicker = URLDecoder.decode(ticker, StandardCharsets.UTF_8.name());
                query = "fin-streamer[data-symbol="" + formattedTicker + ""][data-field="regularMarketPrice"]";
                docElement = doc.selectFirst(query);
                if (docElement == null) {
                    throw new IOException("Cannot find " + query);
                }
                final String priceStr = docElement.attributes().get("data-value");
                quotePrice.setPrice(Double.parseDouble(priceStr));
                quotePrice.setTradeDate(Constants.MISSINGDATE);
                quotePrice.setTradeDateInt(19000101);
                query = "section.container[data-testid="quote-hdr"] span.exchange > span:last-child";
                docElement = doc.selectFirst(query);
                if (docElement == null) {
                    throw new IOException("Cannot find currency element using: " + query);
                }
                quotePrice.setCurrency(docElement.textNodes().getLast().text());
            } catch (IOException e) {
                throw new IOException("Cannot parse response for symbol=" + ticker + e.getMessage(), e);
            } catch (NullPointerException e2) {
                throw new IOException("Cannot parse response for symbol=" + ticker + e2.getMessage(), e2);
            } catch (NumberFormatException e3) {
                throw new IOException("Cannot parse response for symbol=" + ticker + e3.getMessage(), e3);
            }
        }
    

    of course you will have to rebuild the extension and this only fixes the basic yahoo quote loader (i'm not sure if any of the other exchange fetchers are also broken).

    GL

    • Edit

24. 24 Posted by dwg on 20 Apr, 2024 05:21 AM

    

    The developer of Quote Loader is researching what changes will be required to get this extensions working with Yahoo again, he has said it will require some time.

    The Quotes and Exchange rate extension is maintained by TiK themselves, I am not aware how far they may have looked into the issue at this stage.

    • Edit

25. 25 Posted by Mike Bray (Quot... on 20 Apr, 2024 07:03 AM

    

    There is one major issue, the data is not retrieved until the JavaScript is executed. QL at the moment does not execute JavaScript. If you use a browser to retrieve the page and then look at the source you see the source after JS has been executed,

    • Edit

26. 26 Posted by anonymous on 20 Apr, 2024 04:19 PM

    

    maybe you are looking at something else but a page like this: https://finance.yahoo.com/quote/META
    is still server-side rendered (load it with JS disabled in chrome or firefox dev tools) and you can capture all the tags as before (with a few changes as i mentioned above)

    of course you can also query the API directly and fetch all the data from there: https://query1.finance.yahoo.com/v8/finance/chart/NVDA
    e.g. to fetch the last closing price, get this attribute from the response: chart.result[0].meta.regularMarketPrice

    tc

    • Edit

27. 27 Posted by clarkcm2017 on 21 Apr, 2024 06:21 PM

    

    I have used Mike Bray's original suggestion (at the to of this discussion) to switch to FT and putting failing symbols in the "Alt Ticker Box" and it works well for me. I just had to realize that since I was on version 3053 I didn't have a "Alt Ticker box" in that version. After upgrading to the latest version (3204) all is good.

    Perhaps this is a bit obvious to all the experts on this thread but there may be one or two others , like me, who were looking for the missing box!!!!

    • Edit

28. 28 Posted by derekkent23 on 21 Apr, 2024 06:34 PM

    

    I am not support staff, just a user.
    Hi clarkcm2017

    Well spotted.

    Its worth subscribing to this discussion
    https://infinitekind.tenderapp.com/discussions/investments/7595-quo...

    to get a emails which will include notification of when new builds of Quote Loader become available.

    Have Fun, Stay Safe

    • Edit

29. 29 Posted by davidwoh on 21 Apr, 2024 11:40 PM

    

    I guess I am slow. I am using MD 2024.1 (5118) and QL 3204. I see the Alt Ticker column on QL but do not see how to populate this with values. I am double clicking on the Ticker to get the MS Currency/Security History dialog and select Edit. I can change the ticker but do not see a way to enter an Alt Ticker. I have tried everything that I can think of in QL but have also failed.

    • Edit

30. 30 Posted by DrgnMastr on 21 Apr, 2024 11:44 PM

    

    Make sure that the first thing you do with that stock is to set the source to FT. Once it is set, you should be able to access the Alt Ticker.

    • Edit