append the crumb to the URL (Fix #206, #209, #211 and #225)

[gilbertoca]

append the crumb to the URL
Fix #206, #209, #211 and #225

[v3rtumnus]

Hi @gilbertoca ,

are you sure that this change alone fixes the problem? At least, I had to also change the cookie retrieval mechanism: v3rtumnus@a961b19

[gilbertoca]

@v3rtumnus Yes!
I have to say I don't know if the API make any locale request, I'm from Brazil, and I have no complex logic on the call:

        for (AtivoCarteira ativo : carteira) {
            Stock stock = null;
            try {
                final String sticker = ativo.getPapel() + ".SA";
                logger.debug("O simbolo no YahooFiance - {}", sticker);
                stock = YahooFinance.get(sticker);
            } catch (IOException ex) {
                logger.debug("Erro GRAVE", ex);
            }
            if (stock != null) {
                ativo.setStockPrice(stock.getQuote().getPrice());
                ativo.setStockDividend(stock.getDividend().getAnnualYieldPercent());
                ativo.setStockPsr(stock.getStats().getPriceSales());
                ativo.setStockPe(stock.getStats().getPe());
            }

[v3rtumnus]

Ah this could actually explain it because I am from the EU and had problems with the cookie consent banner (maybe this one is not shown when visiting Yahoo outside of EU)

[vkmguy]

I am trying to access the historical data and getting this error, I am at Polish Region(EU) too
java.io.IOException: Server returned HTTP response code: 401 for URL: https://query1.finance.yahoo.com/v7/finance/quote?symbols=GOOG

[chungyeong]

still got an error "Server returned HTTP response code: 429 for URL: https://query1.finance.yahoo.com/v1/test/getcrumb"

[code-monkey-101]

Just as a side note, by accident I discovered today that with Java 21 I got 404 (or 401, don't remember) but Java 1.8 worked fine. Maybe Java 21 doesn't allow you to override the user agent and Yahoo picks up on that.

[bnsd55]

Hey
In addition to changes that were made in this PR, cookie name wasn't accurate, code was looking for B= cookie while it has been changed to A1S=, so I changed it in CrumbManager.java and it worked. No errors, running on Java 17, outside of USA.

[bnsd55]

Change B=.* in line 56 to A1S=.*, that worked for me.

[gilbertoca]

Thank you @bnsd55 !

On my side, I was getting this exception

Caused by: java.io.IOException: Server returned HTTP response code: 503 for URL: https://finance.yahoo.com/quote/%5EGSPC/options

[foxhound91]

@sstrickx could you pls approve and merge, we need this release

[foxhound91]

@gilbertoca this ain't working, try running this test in your local

public class CrumbManagerTest {

    @Test
    public void testGetCrumb() throws IOException {
        assertNotNull(CrumbManager.getCrumb());
    }

}

[gilbertoca]

@foxhound91, it is working locally