Http Client Issue

[kkz6]

Laravel Version

11.9

PHP Version

8.3

Database Driver & Version

No response

Description

I am trying to use HTTP client in a do while loop for pagination. Somehow after calling the first url even after using the updating the url is it calling the same url in the first instance of the loop.

Steps To Reproduce

This is where i setting up some basic things.

public function __construct(protected SourceControl $sourceControl)
    {
        $this->http = Http::withToken($sourceControl->access_token)
            ->retry(2, 0, function (Exception $exception, PendingRequest $request) {
                if (! $exception instanceof RequestException || $exception->response->status() !== 401) {
                    return false;
                }

                $request->withToken($this->getNewToken());

                return true;
            })
            ->acceptJson()
            ->asJson()
            ->timeout(15)
            ->throw();
    }

Here i call the paginated urls.

public function getPaginatedData(string $url): true
    {
        $currentUrl = $url;

        while (!!$currentUrl) {
            $response = $this->http->get($currentUrl, [
                'role'   => 'member',
                'pagelen' => 20
            ]);
            $currentUrl = $response->json('next');
            dump($response->json());
        }

        return true;
    }

even though we get he current url and the new url is passed. The first loop url is getting used.

But,

If I directly use like Http::get then i am not getting this issue. This is really confusing.

[rodrigopedra]

1. Can you provide a sample from dump($response->json())?
2. Can you post a public repo that makes it easier to reproduce?

[kkz6]

1. I was trying it in my private project and found this issue. If I could create a public repo with the same code I will do.
2. the dump response would be some like

If next url is available then
$data = ['values' => [], 'next' => 'example.test?page=2']

Now as per the code in the next iteration of the loop it has to call the url of

'example.test?page=2'

But it calls the url
'example.test'

and so the next url is still present and it will be the same url and thus ends in an infinite loop.

[rodrigopedra]

If I could create a public repo with the same code I will do.

You can create a second endpoint on the sample public repo that mimics the external endpoint.

[rodrigopedra]

Here, I made it for you:

https://github.com/rodrigopedra/bug-report

Instructions are on README.md. The relevant code is in ./routes/web.php.

The issue is that you are using both a query parameter in the URL (?page=2) and sending query parameters through the request's options (['role' => 'member','pagelen' => 20]).

When the options are populated, Laravel does not parse the URL for parameters.

framework/src/Illuminate/Http/Client/PendingRequest.php

Lines 1144 to 1169 in 57d6188

	protected function parseRequestData($method, $url, array $options)
	{
	if ($this->bodyFormat === 'body') {
	return [];
	}
	$laravelData = $options[$this->bodyFormat] ?? $options['query'] ?? [];
	$urlString = Str::of($url);
	if (empty($laravelData) && $method === 'GET' && $urlString->contains('?')) {
	$laravelData = (string) $urlString->after('?');
	}
	if (is_string($laravelData)) {
	parse_str($laravelData, $parsedData);
	$laravelData = is_array($parsedData) ? $parsedData : [];
	}
	if ($laravelData instanceof JsonSerializable) {
	$laravelData = $laravelData->jsonSerialize();
	}
	return is_array($laravelData) ? $laravelData : [];
	}

It seems to me that Laravel should handle both cases (query parameters in the URL and in the request options).

I can't work on a PR right now, as I am leaving for lunch and will be out for the whole day off. If you can, be my guest.

As a workaround, you can parse the next URL query parameters, and add them to the request's options.

[rodrigopedra]

Hey @crynobone any insights why this was moved to a discussion?

When consuming paginated results where the external API sends a URL with a query parameter, the HTTP Client ignores the URL's query parameter if the developer is also setting query parameters on the request $options.

Shouldn't this be considered an issue?

Thanks!

[rodrigopedra]

Also, there is the public repository with reproducible code. Per issue's guidelines.

It is easy to verify the unexpected behavior.

[crynobone]

Based on the replies above this seems to best fit as a topic to discuss further before making a PR. While yes it is we could say it is a bug (a feature is also a process to solve a bug), based on your response it would require a change to the current behavior.

[rodrigopedra]

Thanks for the prompt response.

I don't fully agree on the reasoning, but nonetheless we can agree to disagree.

Should I try a PR? Or is it best to wait on more people to chime in to discuss a potential solution?

[rodrigopedra]

I was working on a PR, but after re-reading your answer:

...this seems to best fit as a topic to discuss further before making a PR.

I will wait for more input from other users.

Or maybe someone else can come up with another PR.

Thanks again, and have a nice day =)

[crynobone]

I'm 50/50 on this changes but it easier to make an argument if this is something that is handled when using Guzzle directly and just not in Laravel's HTTP Client.