Skip to content

Commit c933847

Browse files
author
baton-admin[bot]
committed
chore: update baton-admin doc files
1 parent 51a064b commit c933847

File tree

1 file changed

+118
-0
lines changed

1 file changed

+118
-0
lines changed

.claude/skills/connector/build-pagination.md

Lines changed: 118 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -216,6 +216,124 @@ func (g *groupBuilder) Grants(ctx context.Context, resource *v2.Resource,
216216

217217
---
218218

219+
## Nested Bag Pagination (Bag-within-Bag)
220+
221+
When an outer function uses a Bag to dispatch to inner functions that manage their own Bag, the inner bag's serialized state becomes the `Token` field in the outer bag's `PageState`. This creates a two-layer pagination pattern that is easy to get wrong.
222+
223+
### How It Works
224+
225+
```go
226+
// OUTER: dispatches between resource types
227+
func (o *myType) Grants(ctx context.Context, resource *v2.Resource,
228+
attrs resource2.SyncOpAttrs) ([]*v2.Grant, *resource2.SyncOpResults, error) {
229+
230+
bag := &pagination.Bag{}
231+
bag.Unmarshal(token.Token)
232+
if bag.Current() == nil {
233+
bag.Push(pagination.PageState{ResourceTypeID: "users"})
234+
bag.Push(pagination.PageState{ResourceTypeID: "groups"})
235+
}
236+
237+
page := bag.PageToken() // This is the INNER bag's serialized state
238+
239+
switch bag.ResourceTypeID() {
240+
case "users":
241+
rv, nextPage, annos, err = o.userGrants(ctx, resource, attrs, page)
242+
case "groups":
243+
rv, nextPage, annos, err = o.groupGrants(ctx, resource, attrs, page)
244+
}
245+
246+
// nextPage is the inner bag's Marshal() output.
247+
// When nextPage == "", bag.Next("") pops current type, advances to next.
248+
// When nextPage != "", bag.Next(nextPage) keeps current type with updated token.
249+
bag.Next(nextPage)
250+
251+
pageToken, _ := bag.Marshal()
252+
return rv, &resource2.SyncOpResults{NextPageToken: pageToken}, nil
253+
}
254+
```
255+
256+
### The Ghost State Bug
257+
258+
`pagination.Bag.Marshal()` returns `""` (empty) only when `currentState` is nil. A `PageState` with all zero-value fields (`{Token: "", ResourceID: ""}`) is **non-nil** and serializes to `{"states":[],"current_state":{}}`.
259+
260+
This non-empty string causes the outer bag to think there's still work to do, keeping the current resource type alive instead of advancing to the next one. The inner function then re-initializes from scratch, re-fetching from the beginning — an infinite loop.
261+
262+
```
263+
Inner bag state: {Token: "", ResourceID: ""} (all empty, but non-nil)
264+
Inner Marshal(): {"states":[],"current_state":{}} (non-empty string!)
265+
Outer bag sees: non-empty token → keep "users" type active
266+
Next call: inner Unmarshal → Current() != nil → skip init
267+
ResourceID == "" → fetch API page with Token == "" → page 1 again
268+
INFINITE LOOP
269+
```
270+
271+
### How Ghost States Appear: Stale State in Stack
272+
273+
When you push per-item states on top of a page state, the page state gets buried in the stack. After all items are popped, the page state resurfaces:
274+
275+
```go
276+
// WRONG — page state survives in the stack after all items are popped
277+
for _, user := range users {
278+
if needsExtraProcessing(user) {
279+
bag.Push(pagination.PageState{ // Pushes ON TOP of the page state
280+
ResourceID: user.ID,
281+
})
282+
}
283+
}
284+
285+
// This Pop removes the last pushed user, NOT the page state!
286+
bag.Pop()
287+
if nextPage != "" {
288+
bag.Push(pagination.PageState{Token: nextPage})
289+
}
290+
```
291+
292+
After all users are individually popped in later calls, the original page state `{Token: "", ResourceID: ""}` becomes current again → ghost state → infinite loop.
293+
294+
### Correct Pattern: Pop Page State Before Pushing Items
295+
296+
Pop the consumed page state first, then push the next page (if any), then push per-item states on top. This way the consumed state is gone, and items sit above the next page in the stack.
297+
298+
```go
299+
// CORRECT — Pop consumed state BEFORE pushing anything
300+
bag.Pop() // Remove the page state we just consumed
301+
302+
if nextPage != "" {
303+
bag.Push(pagination.PageState{
304+
Token: nextPage,
305+
ResourceID: "", // Next API page, sits at bottom of stack
306+
})
307+
}
308+
309+
// Now push per-item states on top (processed first, LIFO)
310+
for _, user := range users {
311+
if needsExtraProcessing(user) {
312+
bag.Push(pagination.PageState{
313+
ResourceID: user.ID,
314+
})
315+
}
316+
}
317+
318+
nextPageToken, err := bag.Marshal()
319+
return rv, nextPageToken, annos, nil
320+
```
321+
322+
This guarantees:
323+
- The consumed page state is gone before any items are pushed
324+
- Next page (if any) sits below items, processed after all items are done
325+
- When all items are popped and no next page exists, the bag is truly empty → `Marshal()` returns `""` → outer bag advances
326+
327+
### Invariant: Never Leave All-Empty PageState in the Bag
328+
329+
Every `PageState` in the bag should have at least one non-empty field. If `Marshal()` can produce a non-empty string when there's no real work left, the outer bag will loop.
330+
331+
- Page states: must have non-empty `Token` (guarded by `if nextPage != ""`)
332+
- Item states: must have non-empty `ResourceID` (from the item being processed)
333+
- Initial states (`{Token: "", ResourceID: ""}`) must be popped before returning
334+
335+
---
336+
219337
## Page Size Selection
220338

221339
| API Limit | Recommendation |

0 commit comments

Comments
 (0)