|
| 1 | +# concepts-sync-lifecycle |
| 2 | + |
| 3 | +How connectors sync data to ConductorOne. |
| 4 | + |
| 5 | +--- |
| 6 | + |
| 7 | +## SDK Orchestration |
| 8 | + |
| 9 | +The SDK uses inversion of control. Connectors implement interfaces; SDK orchestrates execution. |
| 10 | + |
| 11 | +``` |
| 12 | +SDK calls connector methods in phases: |
| 13 | +1. ResourceType() - once per type, learn metadata |
| 14 | +2. List() - paginated, fetch all resources |
| 15 | +3. Entitlements() - once per resource, fetch permissions |
| 16 | +4. Grants() - once per resource, fetch assignments |
| 17 | +``` |
| 18 | + |
| 19 | +The connector never controls flow. SDK batches operations, builds access graphs, handles checkpointing. |
| 20 | + |
| 21 | +--- |
| 22 | + |
| 23 | +## Four Sync Phases |
| 24 | + |
| 25 | +### Phase 1: Resource Types |
| 26 | + |
| 27 | +```go |
| 28 | +func (u *userBuilder) ResourceType(ctx context.Context) *v2.ResourceType { |
| 29 | + return &v2.ResourceType{ |
| 30 | + Id: "user", |
| 31 | + DisplayName: "User", |
| 32 | + Traits: []v2.ResourceType_Trait{v2.ResourceType_TRAIT_USER}, |
| 33 | + } |
| 34 | +} |
| 35 | +``` |
| 36 | + |
| 37 | +Called once per resource type. Returns metadata including traits. |
| 38 | + |
| 39 | +### Phase 2: List Resources |
| 40 | + |
| 41 | +```go |
| 42 | +func (u *userBuilder) List(ctx context.Context, parentID *v2.ResourceId, |
| 43 | + token *pagination.Token) ([]*v2.Resource, string, annotations.Annotations, error) |
| 44 | +``` |
| 45 | + |
| 46 | +Called repeatedly with pagination tokens until empty token returned. Must handle: |
| 47 | +- Pagination via token parameter |
| 48 | +- Parent resources (for hierarchical data) |
| 49 | +- Annotations (rate limits, metadata) |
| 50 | + |
| 51 | +### Phase 3: Entitlements |
| 52 | + |
| 53 | +```go |
| 54 | +func (g *groupBuilder) Entitlements(ctx context.Context, resource *v2.Resource, |
| 55 | + token *pagination.Token) ([]*v2.Entitlement, string, annotations.Annotations, error) |
| 56 | +``` |
| 57 | + |
| 58 | +Called once per resource discovered in Phase 2. Returns what permissions exist on this resource. |
| 59 | + |
| 60 | +Example: A group has "member" entitlement that can be granted to users. |
| 61 | + |
| 62 | +### Phase 4: Grants |
| 63 | + |
| 64 | +```go |
| 65 | +func (g *groupBuilder) Grants(ctx context.Context, resource *v2.Resource, |
| 66 | + token *pagination.Token) ([]*v2.Grant, string, annotations.Annotations, error) |
| 67 | +``` |
| 68 | + |
| 69 | +Called once per resource. Returns who has which entitlements. |
| 70 | + |
| 71 | +Example: User "alice" has "member" entitlement on group "admins". |
| 72 | + |
| 73 | +--- |
| 74 | + |
| 75 | +## Checkpointing |
| 76 | + |
| 77 | +SDK checkpoints every 10 seconds during sync. If interrupted: |
| 78 | +- Sync resumes from last checkpoint |
| 79 | +- Connector receives pagination token from checkpoint |
| 80 | +- No need to restart from zero |
| 81 | + |
| 82 | +This is why pagination must be stateless - all state is in the token. |
| 83 | + |
| 84 | +--- |
| 85 | + |
| 86 | +## Stateless Requirement |
| 87 | + |
| 88 | +Connectors must be stateless: |
| 89 | +- No global variables |
| 90 | +- No instance state between calls |
| 91 | +- All context in method parameters |
| 92 | +- Pagination tokens are opaque (SDK manages) |
| 93 | + |
| 94 | +**Rationale:** Connectors may run in Lambda, may be interrupted, may resume on different instance. |
| 95 | + |
| 96 | +--- |
| 97 | + |
| 98 | +## Data Flow Summary |
| 99 | + |
| 100 | +``` |
| 101 | +External API -> Connector.List() -> Resources |
| 102 | + | |
| 103 | + v |
| 104 | + Connector.Entitlements() -> Entitlements |
| 105 | + | |
| 106 | + v |
| 107 | + Connector.Grants() -> Grants |
| 108 | + | |
| 109 | + v |
| 110 | + SDK builds access graph |
| 111 | + | |
| 112 | + v |
| 113 | + c1z file (SQLite + gzip) |
| 114 | + | |
| 115 | + v |
| 116 | + ConductorOne platform |
| 117 | +``` |
0 commit comments