Skip to content

fix(account): prevent infinite pagination loop and duplicate results#6951

Open
AleksandrKls wants to merge 2 commits into
cloudflare:mainfrom
AleksandrKls:fix/cloudflare-account-datasource-loop
Open

fix(account): prevent infinite pagination loop and duplicate results#6951
AleksandrKls wants to merge 2 commits into
cloudflare:mainfrom
AleksandrKls:fix/cloudflare-account-datasource-loop

Conversation

@AleksandrKls

@AleksandrKls AleksandrKls commented Mar 22, 2026

Copy link
Copy Markdown

Changes being requested

Fix infinite/long pagination loop and duplicate results in cloudflare_accounts datasource.

When using an Account API token, the Cloudflare /accounts API may return the same page repeatedly even when result_info.total_pages = 1.

Example behavior observed:

  • Requests to /accounts?page=2+ return the same account as page 1
  • Pagination does not terminate
  • Provider continues calling GetNextPage() until max_items (default 1000)
  • Results contain many duplicates of the same account
  • terraform plan may take several minutes

This PR adds safeguards to handle non-progressing pagination:

  • Deduplicate results by account.id
  • Stop pagination when:
    • page content repeats (same length + same first ID)
    • no new unique accounts are added

This ensures fast execution and prevents duplicate results while preserving normal behavior for valid pagination.

Related to #5298


Acceptance test run results

  • I have added or updated acceptance tests for my changes
  • I have run acceptance tests for my changes and included the results below

Steps to run acceptance tests

N/A – no acceptance tests added (bug is specific to Account API token behavior and requires specific account/token setup to reproduce)

Test output

N/A


Additional context & links

Direct API reproduction using Account API token. Will be returned 5 duplicates:

for p in 1 2 3 4 5; do
  echo "===== PAGE $p ====="
  curl -sS "https://api.cloudflare.com/client/v4/accounts?page=$p&per_page=1" \
    -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \
    | jq '{page: .result_info.page, per_page: .result_info.per_page, count: (.result | length), total_count: .result_info.total_count, total_pages: .result_info.total_pages, ids: [.result[]?.id]}'
done

…in cloudflare_accounts datasource

Stop pagination when no progress is detected or when pages repeat, and deduplicate accounts by ID.

This prevents long-running terraform plan (up to several minutes) and duplicate results when using account API tokens, where the Cloudflare /accounts API may return the same page repeatedly beyond total_pages.
@AleksandrKls
AleksandrKls requested a review from a team as a code owner March 22, 2026 17:31
@AleksandrKls AleksandrKls changed the title fix: prevent infinite pagination loop and duplicate results fix(account): prevent infinite pagination loop and duplicate results Mar 24, 2026
@jhutchings1

Copy link
Copy Markdown
Collaborator

I have pinged our engineering team to look at this PR.

rotem-cloud
rotem-cloud previously approved these changes Mar 30, 2026
@rotem-cloud
rotem-cloud self-requested a review March 30, 2026 20:44
@rotem-cloud
rotem-cloud dismissed their stale review March 30, 2026 20:44

accidental approval before test

@AleksandrKls

Copy link
Copy Markdown
Author

@rotem-cloud Sorry, I haven't created pull requests in Open Source before. Just to clarify, are you expecting me to write tests or will you test them yourself?

@AleksandrKls

Copy link
Copy Markdown
Author

@jhutchings1 Could you please clarify the details regarding my question above?

@mgirouard
mgirouard requested a review from a team April 6, 2026 18:14
@rupalims

rupalims commented Apr 6, 2026

Copy link
Copy Markdown

Hi @AleksandrKls
Our team has some question/concern regarding the MR.
We are standardising pagination scheme.
We have community triage.
Thursday, April 9⋅8:30 – 9:00am PST.
Weekly on Thursday

Join with Google Meet
meet.google.com/cey-wths-axu
Join by phone
‪(US) +1 402-986-3236‬ PIN: ‪673 312 812‬#

Please join to discuss about this

@AleksandrKls

AleksandrKls commented Apr 7, 2026

Copy link
Copy Markdown
Author

I don't want to participate in meetings, but I can answer all your questions in text format.

I'd also like to point out that we have a 10-hour time difference. I live in the UTC +3 time zone

@musa-cf

musa-cf commented Apr 8, 2026

Copy link
Copy Markdown
Contributor

Hi @AleksandrKls, I'm looking into this and we suspect the root cause actually lies upstream in our Go SDK, so I'm exploring the option to patch it there instead. I'll circle back with an update this week. Thanks for opening the PR and providing these details! It helps a ton.

@musa-cf

musa-cf commented Apr 8, 2026

Copy link
Copy Markdown
Contributor

Could you please clarify the details regarding my question above?

As for testing the PR yourself, you're fine. I'll make sure we get the reproduction and test coverage sorted.

@AleksandrKls

Copy link
Copy Markdown
Author

Could you please clarify the details regarding my question above?

As for testing the PR yourself, you're fine. I'll make sure we get the reproduction and test coverage sorted.

It’s not about your Go SDK. The issue lies in how the API behaves when an account API token is used.

You can take my curl request that I attached, send an HTTP request using an account API token (not to be confused with a user token), and you’ll observe the same behavior as with the data resource

@musa-cf

musa-cf commented Apr 10, 2026

Copy link
Copy Markdown
Contributor

I've forwarded that info to the IAM team who owns this API to get it sorted in the API layer. Thanks @AleksandrKls

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants