client.datasets() returns only results from the first page of the response.

[shaiabir]

client.datasets() returns only results from the first page of the response.

when sending a request that results in a long response only the first page is returned unless page=2 for instance is transferred as *kwargs, then you would get the specified page. however there is no way of knowing before hand how many pages will be returned. some sort of pagination mechanism is needed.

    response = client.datasets(
        simulation_round="ISIMIP3b",
        product="InputData",
        climate_forcing=["gfdl-esm4"],
        climate_scenario=["historical", "ssp126", "ssp370"],
        climate_variable=["hurs", "sfcwind", "tasmax", "tasmin", "rsds", "pr"]

[jochenklar]

Hi @shaimarvinblue , I could add a mechanism to retrieve all pages before returning. I think I would need some kind of limit though. Would that match your expectation?

What you already can do is to user page_size=20 or page=2 to control the pagination.

[shaiabir]

yes I guess inputing something like page_size=20000 would work for me now. however a pagination method would be nice as for the current state you are unable to know easily that there where more files in the next page.
Thanks for the super fast response